Device state management

ABSTRACT

A computing environment is disclosed that receives from devices requests directed toward services accessible in the environment, and that forwards communications from services in the environment to devices registered with the environment. During a registration process at the environment, devices are assigned a device identifier that is used to identify and authenticate each particular device and requests communicated from and to the device via the environment. The computing environment maintains state information for each device that has been registered with the system. As the device interacts with the system, the state information is updated to reflect the changes in the device. When requests to perform functions are received from devices, the computing environment determines for the particular device and the particular function requested what processing needs to be performed by the environment in response to the request.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/788,356, filed Jun. 30, 2015, the contents of which is incorporatedherein by reference in its entirety.

BACKGROUND

This application is a continuation of U.S. patent application Ser. No.14/788,356, filed Jun. 30, 2015, the contents of which is incorporatedherein by reference in its entirety.

Electronic devices pervade seemingly every aspect of society. During thecourse of a normal day, it is not atypical for a person to use a smartphone, a tablet device, and laptop computer. Our automobiles rely uponelectronic systems to control and monitor most every feature andoperation. Modern home appliances such as, for example, washers, dryers,and refrigerators are driven and controlled by electronic systems.Manufacturing facilities, building heating and cooling systems, and evenfarming equipment now rely upon electronic sensors and control systems.

Advancements in communication technologies have allowed for even thesimplest of these electronic devices to communicate with other devicesand systems. For example, an electronic device in a manufacturing systemmay monitor various aspects of the manufacturing process and communicatemonitoring data to other devices in the manufacturing system. Similarly,electronic sensors embedded in a building control system may monitor andcommunicate details regarding operation of the building's heating,cooling, and ventilation systems. Even home appliances and lightswitches offer the possibility of being configured with communicationcapabilities for purposes of transmitting status and receiving externalcontrols.

BRIEF DESCRIPTION OF DRAWINGS

The following description of the illustrative embodiments may be betterunderstood when read in conjunction with the appended drawings. It isunderstood that potential embodiments of the disclosed systems andmethods are not limited to those depicted.

FIG. 1 depicts an example computer networking architecture for providingdevice access to network services.

FIG. 2 depicts an example computer networking architecture for providingdevice access to network services.

FIG. 3 depicts an example computer networking architecture for providingdevice access to network services.

FIG. 4 depicts an example computer networking architecture for providingdevice access to network services.

FIG. 5 depicts a flow diagram of an example process for registeringdevices.

FIG. 6 depicts a flow diagram of an example process for associatingentities with devices.

FIG. 7 depicts a flow diagram of an example process for processingdevice connections.

FIG. 8 depicts a flow diagram of an example process for processingdevice requests for server functions.

FIG. 9 depicts a flow diagram of an example process for processingserver requests for device functions.

FIG. 10 depicts a flow diagram of an example process for processingaggregated data.

FIG. 11 depicts an example computer networking architecture for devicestate management.

FIG. 12 depicts an example system for requesting a state change to adevice.

FIG. 13 depicts an example system for receiving and processing a statechange indication from a device.

FIG. 14 is a diagram depicting example actual state and desired stateread requests.

FIGS. 15A-B depict flow diagrams of example processes for managingdevice states.

FIG. 16 depicts a flow diagram of an example process for responding to arequest for device state information.

FIG. 17 depicts a flow diagram of an example process for implementing avirtual command.

FIG. 18 depicts an example computing arrangement for implementingservices accessible to an example device.

FIG. 19 depicts an example computing arrangement for implementingservices accessible to an example device.

FIG. 20 depicts an example computing system that may be used in someembodiments.

DETAILED DESCRIPTION

The continued application of communication features to the growinginstallation of electronic devices offers the potential to create anenormous network of addressable devices. Some have referred to this“network” as the Internet of Things (IOT). The possibility of expansivenetworks of communicatively coupled and addressable devices offersexciting opportunities for the creation of new applications thatleverage the devices and the data collected by them.

Applicants disclose herein a device communication environment thatallows for devices to communicate with services accessible via theenvironment and allows for services to communicate with the devices viathe environment. Devices that have been registered with the devicecommunication environment may forward requests to the environment. Forexample, a device such as a dishwasher may communicate a request thatits operating characteristics be communicated to a network servicesserver accessible by the environment. A gateway server, which may be oneof a plurality of gateway servers, may be identified to process theparticular request. The identified gateway sever communicates with adevice registry server to identify an executable function thatcorresponds to the request. For example, the device registry server mayidentify a particular function executable on a particular networkservice that corresponds to the particular request to communicateoperating characteristics by the particular dishwasher The gatewayserver requests that the identified function be communicated to theappropriate network service.

Requests to control or communicate with a device may be received at adevice shadow server from within the environment. For example, a requestto turn on the dishwasher may be received at a device shadow serverwithin the environment. The device shadow server communicates with thedevice registry server to identify an appropriate request format for theintended device. An appropriately formatted request is transmitted tothe gateway server responsible for communicating with the intendedrecipient device. The gateway server communicates the request to thedevice.

According to one aspect of the disclosed embodiments, the devicecommunication environment, which may be referred to as a system orplatform, is adapted to receive requests to register devices with theenvironment. In an example scenario, a request to register a device maybe received from a manufacturer that is interested in having the devicecommunicate with the communication environment. In an example scenario,the request may comprise information identifying the particular devicesuch as, for example, a manufacturer's device serial number, a MACaddress, or other suitable information that is associated with thedevice.

The request may further comprise information for use in authenticatingthe device when it subsequently attempts to access the devicecommunication system. The information for use in authentication maycomprise any data that is suitable for authenticating the device. Forexample, the request may comprise a digital certificate for use inauthenticating the particular device. In an example scenario, theinformation may comprise, for example, a key that may be used inencryption and/or decryption of data. In an example scenario, the keymay be the public key of a public-private key pair. A device securityserver receives the request in the environment, generates a uniquedevice identifier, stores the information from the request with thedevice identifier, and responds with the unique device identifier. In anexample embodiment, the device registration process may further involvegenerating a certificate which, in an example embodiment, may be storedin association with the device identifier. In an alternate embodiment, acertificate may be generated and communicated to the manufacturer, butnot associated with a particular device identifier. When a devicesubsequently attempts to access the device communication environment,the device provides the generated unique device identifier and mayprovide data that has been encrypted using the private key thatcorresponds to the public key. The device security server authenticatesthe particular device by decrypting the data using the public key. In anembodiment, the request from the device may also comprise a certificatethat the device security server may compare to the certificate that wasearlier generated and may have been stored in association with theparticular device.

According to another aspect of the disclosed embodiments, the devicecommunication environment is adapted to receive requests to registerentities, which may be, for example, individuals and/or organizations,as authorized to control or communicate with a particular device. In anexample scenario, a request may be received from an entity (individualor organization) that may have purchased a device from a manufacturerthat has previously registered the device with the environment. Forexample, the device may be a dishwasher, thermostat, or lightingassembly that an individual or organization purchased from themanufacturer. The entity may initiate a request, perhaps using a mobilephone application, to register the device as being associated with theentity. The request identifies the device using the unique identifierassigned during device registration and provides data identifying theparticular entity that is requesting to be associated with the device.The request is received at the device security server and informationassociating the particular entity stored in relation to the deviceidentifier. When the particular entity subsequently attempts tocommunicate data or commands to the particular device, the devicesecurity server uses the information received in the registrationrequest to confirm that the particular entity is authorized tocommunicate with or control the particular device. Conversely, when anunregistered entity attempts to communicate with or control the device,the communication environment uses the information provided duringregistration to deny any such request.

Another recurring impediment to the creation of a device communicationenvironment has been disparate technical characteristics of the numerousdifferent devices that may attempt to connect to the environment. Ofparticular note are the numerous different commands or functions thatvarious different devices may wish to perform. For example, a first setof devices may be programmed to expect data to be stored and retrievedin a first manner, while a second set of devices may be programmed toexpect data to be stored and retrieved in a second manner. Moreover, thevarious electronic devices often use different communication protocols.For example, some electronic devices may communicate using HTTPs (HyperText Transport Protocol Secure), others may communicate using CoAP(Constrained Application Protocol), and still others may communicateusing MQTT (Message Queue Telemetry Transport). The use of disparateprotocols complicates the creation of a single network environment foruse by all devices.

According to an aspect of the disclosed embodiments, the devicecommunication system provides a gateway server for routing data andcommands between electronic devices and a computing environmentcomprising computing resources. Upon receiving a request from anelectronic device, a gateway server queries a distributed table in orderto identify a particular gateway server that is responsible forprocessing requests from the particular device. The identified gatewayserver is programmed to convert requests formatted using functions orcommands and protocols specific to the particular device into functionsor commands and a format usable within a computing environment forproviding computing resources.

In an example embodiment, a plurality of gateway servers are accessiblevia the Internet and available to provide access to computing services.Each gateway server is communicatively coupled with associated computingservers that operate to provide devices with access to a computingenvironment containing computing resources and to provide computingresources in the environment with access to connected devices.

In an example embodiment, each gateway is communicatively coupled with adistributed hash server. The distributed hash server may comprise atable of hash values corresponding to unique identifiers of the devicesthat connect to the computing environment. In an example embodiment, thehash table forms a consistent hash ring with each of the plurality ofgateway servers having a hash value on the ring. The distributed hashserver may be queried to locate the hash value for a particular deviceand a hash value corresponding to a gateway server that may be used forcommunication between the particular device and the computingenvironment.

In an example embodiment, each gateway server may be communicativelycoupled with a device security server comprising information forauthenticating that each particular device is authorized to communicatewith the device communication system. Upon receipt of requests fromdevices, a gateway server may communicate with the device securityserver and with the device to confirm that the device is registered withthe system and not otherwise excluded from communicating with thesystem.

In an example embodiment, each gateway server is also communicativelycoupled with a device registry server. The device registry server isprogrammed with rules or logic for converting specialized devicefunctions or commands received from devices in particular communicationprotocols such as, for example HTTPS, MQTT, CoAP, into functions orcommands and protocols that are understood by other of the servers inthe computing environment that provide the requested services. Forexample, the device registry server may be programmed with rules orlogic for converting a request to store a file from a particular deviceinto a storage function offered by a web services server. The deviceregistry server is similarly programmed to receive requests from serverswithin the computing environment that provide services and to convertthose requests into commands and protocols understood by connectingdevices. For example, the device registry server may be programmed toreceive a request to change the operating status of a particular deviceand convert that request into a function and format that is executableby the particular device to change operating status.

Each gateway server may further be communicatively coupled with a deviceshadowing server. The device shadowing server comprises data specifyingone or more states for each device that is connected to the computingenvironment. In an example scenario, the device shadowing server mayrequest that the gateway server communicate a status change to aconnected device.

Each gateway server may be communicatively coupled with one or more“cloud services” or “web services” servers. Cloud/web services serversare programmed to provide functions or services to devices that accessthe computing environment. For example, web services servers may providedata storage and retrieval services and/or provide access to particularprocessing functionality. Web services servers may be those used toprovide any suitable cloud-based or Web-based services.

Each gateway server may still further be communicatively coupled with adispatcher server. The dispatcher server is programmed to provide aninterface between the gateway server and the other servers in thecomputing environment. A gateway server forwards to the dispatcherserver one or more requests to execute functions that correspond to therequest received from the device. The dispatcher server communicates therequests to execute functions to the appropriate web server servers andcommunicates responsive information to the gateway server which mayforward the responsive information to the requesting device.

According to an aspect of the disclosed embodiments, the gateway serverprocesses requests from electronic devices for access to computingresources provided by the computing environment. In an example scenario,a request is received from a device at one of a plurality of gatewayservers. The request may have been forwarded by a load balancing serverthat may have information provisioned therein identifying a particulargateway server to which requests from the particular device should beforwarded. It will be appreciated that the request may be received inany of several different communication protocols including, for exampleHTTPS, CoAP, and MQTT and further that the request may specify afunction or capability that is particular to requests from theparticular device or device type.

Upon receipt of requests from devices, the gateway server that wasidentified by the load balancer server and at which the request wasreceived may communicate with the device security server and with thedevice to confirm that the device is registered with the system and nototherwise excluded from communicating with the system. In an exampleembodiment, the gateway server may forward to the device security servera device identifier received with the request along with any informationthat may be used to authenticate the device. The device security servermay perform a handshake with the device via the gateway server in orderto confirm that the device is, in fact, registered and eligible tocommunicate with the communication system.

Assuming the device is registered and authorized to communicate with thedevice communication system, the gateway server communicates the requestthat was received from the device to the device registry server. Thedevice registry server queries its rules and logic in order to identifyprocessing functions or commands executable by resources in thecomputing environment that correspond to the particular functions orcommands received from the particular device. In other words, the deviceregistry server converts the function or command received from thedevice into functions or commands that may be implemented in thecomputing environment. The device registry server returns informationidentifying the corresponding functions or commands to the identifiedgateway server.

The identified gateway server communicates the identified processingfunctions or commands to the dispatcher server. The dispatcher servercommunicates requests to perform the identified functions or commands tothe relevant resources in the computing environment. In an examplescenario, the dispatcher server communicates the commands to one or moreweb services servers that provide data storage, retrieval, andprocessing features. The dispatcher server communicates any results fromthe processing to the identified gateway sever which communicates aresponse to the requesting device.

According to another aspect of the disclosed embodiments, the discloseddevice communication environment is adapted to receive requests ormessages from the computing environment and transmit correspondingrequests or messages to the appropriate device. In an exampleembodiment, the environment is adapted to determine whether the requestsdirected to a particular device are authorized. For example, when arequest is received from a web services system or a system outside ofthe environment, the request may be routed to the device security serverwhich evaluates the request in order to determine if the request isauthorized. In an example scenario, the device security server evaluatesthe request in order to determine whether the request is from an entity(individual or organization) authorized to perform the requested controlof the device. For example, the device security server comparesinformation received with the device indicating who the generated therequest with information identifying entities that previously registeredas being authorized to control the particular device. In the scenariowhere the information received with a request indicates the request isfrom an entity authorized to communicate with or control the device, thedevice security server forwards the request for further processing. Inan example scenario, the request may be forwarded to a device shadowserver, if the request requires processing at the device shadow server,or to a gateway server.

In an example scenario, the request or message is received at a gatewayserver from a computing system in the computing environment. Forexample, a request to change the operating status of a device may bereceived from a device shadowing server. The request may specify theparticular device by specifying a particular connection.

The gateway server at which the request is received identifies one ofthe plurality of gateway servers as responsible for processing requeststo the particular device. In an example embodiment, the gateway serverat which the request is received queries the distributed hash server toidentify one of the plurality of gateway servers as corresponding to theparticular connection or device to which the request is directed. In anexample embodiment, the distributed hash server queries a consistenthash ring to identify the one of the plurality of gateway seversresponsible for the particular device.

The identified gateway server formats a request or message formatted forcommunication to and processing by the target device. In an exampleembodiment, the identified gateway server may request that the deviceregistry server query its rules database to identify a command orfunction suitable for the processing by the device but corresponding tothat received by the identified gateway server. Once the identifiedgateway server identifies the function or command corresponding to thatof the received request, the identified gateway communicates thefunction or command formatted for processing by the target device.

While the above discussion provides detail regarding some example devicegateway features of the described techniques, it is noted that thedescribed techniques also provide a number of device state managementfeatures that may improve upon existing technologies. In particular, onedrawback of existing technologies is that connected devices may havevarious associated operational and functional limitations that maynegatively impact the ability of cloud or other distributed computingapplications to interact with the devices. For example, in order toconserve power, many battery-powered and other devices may disablewireless radios when not actively transmitting data or polling forupdates. For these and other reasons, many devices may connect toexternal components only intermittently, while applications may attemptto interact with the devices at any time. Furthermore, as devices andtheir corresponding functionality become increasingly more complex, theproduction and operating costs for the devices may continue to increase.

In order to alleviate these and other concerns, the described techniquesmay employ a device shadowing service that may generate and maintain acached representation of each connected device. Devices may report statechanges and other information to the service, and this reportedinformation may be stored within the device representation. The cachedinformation may then be made available to applications at any time bythe device shadowing service without the need to wake up or otherwisecontact a device itself. Thus, the caching of information by theshadowing service may reduce the frequency and quantity ofcommunications between the devices and external components, therebyreducing device operating costs, such as power consumption costs andnetwork communications costs.

In some cases, when devices report state changes, the state changes maytrigger various operations to be performed by the service. For example,the service may process reported information to identify errors,calculate mean, maximum, and minimum values, and other operations. Asanother example, the service may identify safety concerns and otherproblematic conditions, such as multiple readings that fall outside of adesired range. In some cases, the ability of the service to performadditional processing operations may reduce the amount of processingrequired by the connected devices, thereby reducing cost and complexityof the devices. In some examples, additional operations may be performedusing cloud or other distributed computing functions that are triggeredin response to events. The use of such event-triggered computingfunctions may further reduce costs by consuming computing resources onlyat times when they are invoked.

The device shadowing service may also, for example, process requestsfrom applications to change device state and otherwise control ormanipulate the connected devices. For example, the device shadowingservice may include logic to determine when a desired device statediffers from an actual device state and to issue a command to the deviceto change state based on such a determination. Additionally, the deviceshadowing service may allow virtual device commands to be defined andexposed to applications. These virtual commands may be formulated basedon one or more existing device commands that are understood by thedevice. For example, a virtual “blink” command for a light could includecombinations of existing “on” and “off” commands called in succession togenerate a blinking effect. When a virtual command is issued by anapplication, the command may be processed by the service, by, forexample, invoking an associated event-triggered computing function.

Example Computing Environment

FIG. 1 is a diagram illustrating an example computing environment 110with which devices may communicate. Computing environment 110, which maybe referred to as a device communication environment or system,comprises various resources that are made accessible via gateway 140 todevices 130 that access the gateway via network 120. Devices 130 mayaccess computing environment 110 in order to access services such asdata storage and computing processing features. Services operating inenvironment 110 communicate data and messages to devices 130 in responseto requests from devices and/or in response to computing operationswithin the services.

Computing environment 110 comprises communicatively coupled componentsystems 140, 142, 144, 146, and 150 that operate to provide services todevices 130. Gateway server 140 is programmed to provide an interfacebetween devices 130 and computing environment 110. Gateway server 140receives requests from devices 130 and forwards corresponding data andmessages to the appropriate systems within environment 110. Likewise,when systems within environment 110 attempt to communicate datainstructions to devices 130, gateway server 140 routes those requests tothe correct device.

Gateway server 140 is adapted to communicate with varied devices 130using various different computing and communication capabilities. Forexample, gateway server 140 may be adapted to communicate using eitherTCP (Transmission Control Protocol) or UDP (User Datagram Protocol)protocols. Likewise, gateway server 140 may be programmed to receive andcommunicate with devices 130 using any suitable protocol including, forexample, MQTT, CoAP, HTTP, and HTTPS. Gateway server 140 is programmedto convert the data and instructions or messages received fromelectronic devices into a format that may be used by other of the serversystems comprised in computing environment 110. In an example scenario,gateway server 140 may be adapted to convert a message received usingthe HTTPS protocol into a JSON (JavaScript Object Notation) formattedmessage that is suitable for communication to other servers withinenvironment 110.

Gateway server 140 may itself store, or may control the storing, ofinformation regarding the devices 130 that have formed connection to theparticular gateway server and for which the particular gateway server isgenerally relied upon for communications with the device. In an exampleembodiment, gateway server 140 may have stored thereon informationspecifying the particular device such as a device identifier, i.e.,device_id. For each connection established from the particular device,the gateway server 140 may also maintain information identifying theconnection. For example, a connection identifier, i.e., connection_id,may be generated and stored for each connection established with aparticular device. Information relating the particular connection mayalso be stored. For example, information identifying the particularsocket of the gateway server 140 on which the connection wasestablished, as well as information identifying the particular protocolused by the device on the connection may be stored by gateway server140. Information such as the socket and protocol may be used in order tofacilitate further communications via the particular connection.

In an example embodiment, gateway server 140 is communicatively via anysuitable networking technology with device registry server 142. Deviceregistry server 142 is adapted to track the attributes and capabilitiesof each device 130. In an example embodiment, device registry sever 142is provisioned with information specifying the attributes of devices130. In an example scenario device registry server 142 comprises dataspecifying rules or logic for handling the various requests that may bereceived from devices 130. Device registry server 130 may be programmedto convert specialized device functions or commands received inparticular communication protocols such as, for example HTTPS, MQTT,CoAP, into functions or commands using particular protocols that areunderstood by other of the servers in the computing environment 110. Inan example scenario, device registry server 142 may be provisioned withinformation specifying that upon receipt of a particular request from aparticular device 130, a request should be made to store the payloaddata of the request in a particular network service server 150. Deviceregistry server 142 is similarly programmed to receive requests fromservers 142, 150 and convert those requests into commands and protocolsunderstood by devices 130. In an example scenario, device registryserver 142 may be provisioned with information specifying that uponreceipt of a particular function or command from server 142 for aparticular device or connection, device registry identifies acorresponding function or command for the particular device 130

Device shadowing server 144 maintains state information for eachconnected device 130. In an example embodiment, device shadowing server144 maintains for each device 130 that has connected to the environment110 information specifying a plurality of states. In an examplescenario, device shadowing server 144 may comprise an actual state and adesired state. The actual state represents the existing state of theparticular device 130 as presently known to server 144. In an examplescenario, for example, device shadowing server 144 may identify that theactual state of a light switch device is “on.” The desired staterepresents a state to which it is desired to change the status. In anexample scenario, device shadowing server 144 may identify the desiredstate of a light switch device is “off” Network services server 150 maycommunicate with device shadowing server 144 in order to change thedesired state information. Device shadowing server 144 communicates withdevice gateway 140 in order to communicate requests to update a statusto a particular device 130. For example, device shadowing sever 144 maycommunicate to device gateway 140 that a particular switch device shouldhave its state changed to “off.” Device gateway 140 may, in response,communicate that appropriate command formatted for the particulardevice.

Device security server 146 maintains security-related information fordevices 130 that connect to environment 110. In an example embodiment,device security server 146 is programmed to process requests to registerdevices with environment 110. For example, entities such as, forexample, device manufacturers, may forward requests to register deviceswith environment 110. Device security server 146 receives registrationrequests and assigns unique device identifiers to devices which use thedevice identifiers on subsequent requests to access environment 110.Device security server 146 stores, for each registered device,authentication information that is provided during the deviceregistration process. For example, a request to register a device maycomprise information identifying the device such as a device serialnumber and information for use in authenticating a device. In an examplescenario, the information may comprise a digital certificate and maycomprise a public key of a public key-private key pair. The informationis stored in relation to the assigned device identifier for theparticular device. When the device subsequently attempts to accessenvironment 110, the request may be routed to device security server 146for evaluation. Device security server 146 determines whetherauthentication information provided in the request is consistent withthe authentication information stored in relation to the deviceidentifier and provided during the registration process.

Device security server 146 may be further programmed to process requestto associate particular entities (individuals or organizations) withparticular devices. The device security server 146 is adapted to receiverequests to register entities, which may be, for example, individuals,users, accounts, and/or organizations, as authorized to control orcommunicate with a particular device. In an example scenario, a requestmay be received from an individual or organization that may havepurchased a device from a manufacturer. For example, the device may be adishwasher, thermostat, or lighting assembly that an individual ororganization purchased from the manufacturer. The individual ororganization may initiate a request, perhaps using a mobile phoneapplication, to register the device with the individual or anorganization with which the organization is associated. The request maybe routed to a web services server which may be comprised in environment110 or which communicates the request to the environment 110. Therequest identifies the device and the particular entity (individual ororganization) that is requesting to be associated with the device. In anexample embodiment, the request may comprise the unique deviceidentifier that was assigned when the device was registered with thesystem. The request further may comprise information uniquelyidentifying the entity that is registering as having authority tocommunicate with and/or control the particular device.

Device security server 146 stores the information identifying theparticular entity in relation with the device identifier. When theparticular entity subsequently attempts to control or communicate datato the particular device, the device security server 146 may use theinformation to confirm that the particular entity is authorized tocommunicate with or control the particular device. When an entity thathas not been registered as being authorized to communicate with thedevice attempts to communicate with or control the device, devicesecurity server 146 may use the information stored in the devicesecurity server to deny the request.

Network services server 150 may be any resource or processing serverthat may be used by any of servers 140, 142, 144, 146, 148 in processingrequests from devices 130. In an example scenario, network servicesserver 150 may provide data storage and retrieval services and/oron-demand processing capacity. In an example scenario, network servicesserver 150 may be any of numerous network accessible services including,for example, web or cloud-based services such as, for example, AmazonLambda, Amazon DynamoDB, and/or Amazon Kinesis. In an exampleembodiment, web services server 150 may be specially programmed toprovide particular processing for particular devices 130 and/or groupsof devices 130. For example, a network services server 150 may beprovisioned with software that coordinates the operation of a particularset of devices 130 that control a particular manufacturing operation.

Servers 140, 142, 144, 146 and 150 are communicatively coupled via anysuitable networking hardware and software. For example, the servers maycommunicate via a local area or wide area network.

External system 160 may access environment 110 for any number ofpurposes. In an example scenario, external system 160 may be a systemadapted to forward requests to register devices with environment 110.For example, external system 160 may be a server operated by or for adevice manufacturer that sends requests to environment 110, and devicesecurity server 146 in particular, to register devices 130 for operationwith environment 110. Similarly, external system 160 may be a systemoperated to provide a gateway for entities (individuals ororganizations) to register an ownership or control relationship with aparticular device 130. For example, external system 160 may be a systemoperated by or for a device manufacturer that individuals ororganizations who purchased devices from the manufacturer may access toregister ownership of a purchased device.

Devices 130 may be any devices that may be communicatively coupled via anetwork 120 with computing environment 110. For example, devices 130 maybe computing devices such as smart phones and tablet computers,automobiles, appliances such as washers and driers, industrial sensors,switches, control systems, etc. In an example scenario, each of devices130 may communicate over network 120 to store data reflecting theoperations of the particular device and/or to request processingprovided by, for example, network services server 150. While FIG. 1depicts three devices 130, it will be appreciated that any number ofdevices 130 may access environment 110 via gateway server 110. Furtherit will be appreciated that the devices may employ various differentcommunication protocols. For example, some devices 130 may transportdata using TCP, while others may communicate data using UDP. Somedevices 130 may use MQTT, while others may use CoAP, and still othersmay use HTTPs. It will also be appreciated that each of devices 130 maybe programmed to send and receive particular functions or commands inits requests that are not compatible with other devices or even thesystems within computing environment 110. Gateway server 140 isprogrammed to receive and, if needed, attend to converting such requestsfor processing with computing environment 110.

Communications network 120 may be any type of network that is suitablefor providing communications between devices 130 and computingenvironment 110. Moreover, communications network 120 may comprise acombination of discrete networks which may use different technologies.For example, communications network 120 may comprise local area networks(LANs), wide area networks (WAN's), cellular networks, or combinationsthereof. Communications network 120 may comprise wireless, wireline, orcombination thereof. In an exemplary embodiment, communications network120 comprises the Internet and may additionally comprise any networksadapted to communicate with the Internet. Still further, thecommunications network 120 may make use of any suitable protocols suchas, for example, Web protocols that employ HTTP.

It will be appreciated that there may be any number of devices 130 thataccess computing environment 110. Indeed, billions of electronic devicesexist that may benefit from interfacing with computing environment 110.Moreover, these same devices will be geographically distributed. FIG. 2illustrates an example architecture of a computing environment 110suitable for handling processing associated with large numbers ofdevices 130 which may be geographically distributed. As shown, aplurality of devices 130, which may number in the thousands, millions,or even billions, are communicatively coupled to network 120. In thedisclosed embodiment, computing environment 110 comprises a plurality ofload balancers 210 which operate to route requests from devices 130 to aplurality of gateway servers 110. Generally, load balancers 210 operateto distribute the processing load in a desired fashion amongst theavailable gateway servers 140. Any suitable load balancer may employedincluding, for example, that described in U.S. patent application Ser.No. 13/864,167 titled “Distributed Load Balancer,” the contents of whichare hereby incorporated herein by reference in its entirety. In anexample embodiment, each request from devices 130 is routed by one ormore domain name services (DNS) hosting services in network 120 to aparticular load balancer 210, the internet protocol (IP) addresses ofwhich are registered with the DNS services.

Each load balancer 210 is communicatively coupled with one or moregateway servers 140. Upon receipt of a request from a device, the loadbalancer 210 at which the request is received routes the request to agateway server 210 that has been identified as responsible forprocessing requests from the particular device 130 from which therequest was received. The load balancer 210 at which the request wasreceived may have been provisioned with information indicating thatrequests from particular devices should be routed to particular gatewayservers 210.

Referring to FIG. 2 , each of gateway servers 140 has access via, forexample one or more networks, to an arrangement of computing devicessuch as is illustrated by the environment designated 110 a. For purposesof clarity, the details of computing arrangement 110 a are illustratedin connection with gateway server 140 a, although similar features applyto the other gateway servers as well. Gateway server 140 a iscommunicatively coupled with device registry server 142, device shadowserver 144, device security server 146, and network services server 150which operate consistent with the description above in connection withFIG. 1 . Gateway server 140 a is further communicatively coupled withdistributed hash server 220, discovery failure detection daemon 240, anddispatcher server 230.

Distributed hash server 220 comprises information that allows gatewayservers 140 to identify the particular gateway server that isresponsible for communication with a particular device and, in someinstances, the particular gateway server that is responsible forcommunicating with a particular device on a particular connection. Anysuitable manner of storing information to allow gateway servers toderive the appropriate gateway server may be used. In an exampleembodiment, hash values for devices, connections, and device gatewaysare generated and stored in relation to each other for later querying inorder to identify the particular device gateway that corresponds to aparticular device or connection.

In an example embodiment, a hash value for each device that haspreviously communicated with a gateway in server environment 110 may becomprised in distributed hash server 220. In an example embodiment, thehash values stored in server 220 have been generated by performing ahashing function on a unique identifier corresponding to each of devices130. The unique identifier used in generating the hash value may be anythat is suitable for identifying individual devices. For example, theunique identifier may be a device identifier, i.e., device_id, thatcorresponds to a particular device and which may have been generated bydevice security server 146 during a registration process. The uniqueidentifier may be one that is associated with the physical hardware ofthe device such as, for example, a universally unique identifier (UUID),a machine access control address (MAC address), an IP address, etc. Inan embodiment, unique identifiers may be assigned to devices when theyfirst initiate communication with environment 110. For example, thesystem and, in particular, device security server 146 may generate aunique identifier for the particular device. The unique identifier maybe, for example, an identifier formatted consistent with InternetProtocol version 6. A hashing function, which may be any that issuitable, is performed on the unique identifier and the result stored indistributed hash server 220.

In an example embodiment, distributed hash server 220 comprisesinformation specifying the particular gateway server 140 thatcorresponds to each device 130. In an example embodiment, the hash datais implemented as a consistent hash ring wherein identifiers of gatewayservers 140 are included in the hash ring in addition to hashes of thedevice identifiers. Hash functions of identifiers associated with thegateway servers 140 are computed and the resulting hash values stored ina manner such that the hash value of a gateway server corresponding to aparticular device may be located and the particular gateway serveridentified. For example, the hash values corresponding to gatewayservers may be placed on the ring such that once a hash value for aparticular device is located, locating the hash value of thecorresponding gateway server is performed by moving in a particulardirection, e.g., clockwise, on the ring until reaching the first gatewayhash value. The first gateway server hash located upon moving in aclockwise direction corresponds to the gateway server responsible forresponding to the request.

According to another aspect of an example embodiment, the distributedhash server 220 may further comprise hash values for informationidentifying particular connections with devices. A connection identifiermay be generated by the gateway server 140 or another component serverwhen devices access the system. Each gateway server may maintain theconnection identifier information for the connection established throughthat server. However, the distributed hash server 220 may also beprovisioned with hash values for the connection identifier informationso that all gateway servers may identify the particular gateway serverresponsible for the connection. Accordingly, a hash value for eachconnection identifier may be stored in relation to the hash value of thedevice and the hash value of the gateway server to which the connectionidentifier corresponds. The hash value of the connection information maybe stored in a consistent hash ring as noted above, and may be used toretrieve the corresponding device identifier and/or gateway identifierin a manner similar to that described above.

Discovery failure detection daemon 240 comprises a database of gatewaydevices 140 that are currently active and available for communicatingwith devices. Discovery failure detection daemon 240 may be queried byservers such as device shadow server 144 and/or network services server150 prior to attempting to contact a gateway server 140 for purposes ofcommunicating a message to a device via a gateway server. Performingsuch a query allows servers to identify which servers are/are notavailable.

Dispatcher server 230 is programmed to act as an interface betweengateway server 140 a and the other servers such as device shadow server144 and network services server 150 in the computing environment 110 a.Gateway server 140 a forwards one or more requests to perform functionsor commands that correspond to functions or commands in a requestreceived from device 130 a to dispatcher server 230. Dispatcher server230 communicates a request to perform the identified functions orcommands to the appropriate network services servers 150 andcommunicates responsive information to gateway server 140 a, which mayforward the responsive information to device 130 a.

It will be appreciated that the various computing systems, includingservers, that are described herein, may be implemented in any suitablemanner. For example, the computing systems may be implemented as virtualservers or virtual instances that exist on one or more physical serversas described below in connection with FIGS. 18 and 19 . It will befurther be appreciated that the various servers described in connectionwith FIGS. 1 and 2 may be arranged and distributed across physical andvirtual servers in any suitable manner. In an example embodiment, theservers may be comprised in a single instance of a physical or virtualserver. For example, Gateway server 140, dispatcher 230, device registry142, device shadow server 144, device security server 146, and networkservices server 150 may be comprised on or hosted on a single physicalor virtual server instance. In other embodiments, each of serversgateway server 140, dispatcher 230, device registry 142, device shadowserver 144, device security server 146, and network services server 150may execute on separate physical or virtual server instances. Thecomputing devices/servers and the functionality described herein may bedistributed in any manner among server instances and remain within thescope of intended embodiments.

FIG. 3 depicts component portions of an example embodiment of gatewayserver 140. As shown, in an example embodiment, gateway server 140 maycomprise gateway front end 310 and gateway web service end 320. Gatewayfront end 310 generally corresponds to that portion of the gatewayserver's functionality in interfacing with devices 130. For example,gateway front end 310 both receives and processes requests from devicesand communicates requests to devices 130. Gateway web service interface320 corresponds to that portion of gateway server's functionality usedin interfacing with services within the computing environment 110. Forexample, where a network services server 150 has data or a request fortransmission to a particular device 130, the data and request may bereceived by gateway web service 320.

As illustrated in FIG. 3 , in an example embodiment, dispatcher server230 may be configured as a portion of gateway server 140. Likewise,distributed hash server 220 may be configured as a portion of gatewaysever 140.

It will be appreciated that computing environment 110 depicted in FIGS.1 and 2 , may be arranged in numerous different configurations. FIG. 4depicts another example configuration. In the particular configurationof FIG. 4 , gateway servers 140 located in geographically distantregions 410 are communicatively coupled via overlay network 420 whichmay employ, for example, a backbone networking technology such as bordergateway protocol (BGP) for coordination between servers 140. In such anembodiment, each cluster of gateway servers 140 may form an autonomoussystem with an assigned subnet with an associated IPv6 number. In anexample embodiment, each device 130 that connects to a node in aparticular region 410 may be assigned a unique identifier such as, forexample, an IPv6 address that corresponds to the particular subnetassociated with the particular region 410. In such an embodiment, when adevice with an IPv6 formatted number makes a request, identifying thecorresponding region and gateway is a relatively straightforward routingexercise.

Example Request Processing

In an example embodiment, devices 130 must be registered withenvironment 110 in order to communicate with environment 110. In anexample embodiment, device communication environment 110 maintains adevice identifier for each registered device 130. The device identifier,which may be referred to as a device_id, uniquely identifies aparticular device 130 within environment 110 and may be used indetermining whether to provide access to a particular device. In someembodiments, the process of registering devices with environment 110 mayinvolve provisioning devices with certificates that may be used duringsubsequent communications with environment 110.

The process of registering devices 130 with environment 110, includingassigning device identifiers, may be performed using any suitableprocessing. FIG. 5 depicts a flow chart of an example process forregistering a device 130 with device communication environment 110. Atblock 510, a request is received at device security server 146 toregister a device with the environment 110. The request may be receivedfrom any system that is adapted to communicate a properly formattedrequest to device security server 146. In an example scenario, requeststo assign device identifiers to particular devices may be received from,for example, a web services server 150, which itself may have received arequest from external system 160 which may be maintained by, forexample, a manufacturer of devices. In still another scenario, a requestto assign a device identifier may be received from the device itself.The request may be received using an application programming interface(API) that the device security server 146 makes available for receivingregistration requests.

The request to register a device 130 comprises information about theparticular device which is subsequently associated with the device 130in environment 110. In an example scenario where a device 130 is beingregistered by a manufacturer of the device, the request may comprise aserial number that the manufacturer has associated with the particulardevice. In an alternative scenario, the information about the device maybe a unique identifier that is associated with the physical hardware ofthe device such as, for example, a universally unique identifier (UUID)or a machine access control address (MAC address). In an exampleembodiment, the request may further comprise information that may beused to authenticate the particular device 130 when the device 130subsequently attempts to communicate with communication environment 110.In an example embodiment, the request may comprise a public key of apublic key-private key pair. The public key may be used on subsequentattempts by the device 130 to communicate with environment 110. Moreparticularly, subsequent attempts by the particular device 130 to accesscommunication environment 110 may comprise data encrypted with theprivate key of the key pair. The communication platform 110 uses thepublic key to decrypt the received encrypted data and therebyauthenticate the particular device 110. In an alternative embodiment,the public key may not be provided in the received request, but may begenerated by device security server 146.

In an example scenario, a request to register a device may furthercomprise a certificate signing request (CSR), in response to which,device security server 146 generates and provides a public keycertificate. The CSR may take any suitable form including, for example,a public key cryptography standard (PKCS) number 10. In an exampleembodiment, the request may be formatted as a call to aCreateCertificate API which may take as an argument the certificatesigning request.

In some instances, it may be desired to assign an activation code to adevice which may subsequently be used during activation of the device.In such an embodiment, the activation code may be identified during theprocess of device registration. For example, the manufacturer maycommunicate a device activation code as part of the device registrationprocess so that environment 110 is aware of the code that is required toactivate the particular device. In an example embodiment wherein thereceived request is a formatted as a CreateCertificate API request, theAPI call may further take as an argument an activation code that theorganization that is making the request, e.g., the manufacturer, intendsto be assigned to the particular device. For example, the request mayappear as follows: CreateCertificate(certificateSigningRequest,[activationCode]). In an embodiment where activation codes are employed,but the device registration request does not include an activation code,device security server 146 may generate an activation code as part ofthe device registration process.

It will be appreciated that the request to register a device 130 maycomprise a request to register a plurality of individual devices. Forexample, a request may have originated from an external system 160operated by or for a device manufacturer and may specify a plurality ofdevices that were recently manufactured and which the manufacturerwishes to register. Device security server 146 is programmed to processrequests identifying a plurality of devices using any of severalsuitable techniques including, for example, processing each requestseparately.

Referring to FIG. 5 , at block 512, device security server 146 generatesand/or allocates a particular device identifier for the particulardevice identified in the request. The format and content of the deviceidentifier may be any that is suitable to uniquely identify theparticular device. In an example embodiment, device security server 146may generate a unique identifier consistent with the Internet Protocolversion 6.

In addition, device security server 146 may generate a public keycertificate that illustrates ownership of a public key. For example,device security server 146 may generate a public key certificate inresponse to receiving a certificate signing request as part of thedevice registration. In an example embodiment, the generated certificatemay subsequently be used when the particular device attempts tocommunicate with environment 110 during a TLS certificate exchange.

In an embodiment wherein the registration processing involves assigningactivations codes to devices, and the request to assign an activationcode does not include a code, device security server 146 may generatethe activation code. The generated activation code may have any suitableformat.

At block 514, device security server 146 associates the generated deviceidentifier with the related identifying information for the particulardevice. For example, device security server 146 may store the deviceidentifier in association with the device identifying information (e.g.,device serial number, MAC address, etc.) and any authenticationinformation (e.g., a public key or digital certificate) that wasprovided in the request. Likewise, device security server 146 mayassociate any public key certificate that may have been generated withthe device identifier. Still further, device security server 146 mayassociate the generated device identifier with any activation code thatwas provided by the manufacturer or which may have been created bydevice security server 146. Device security server 146 may form theassociation by storing the device identifier in relation to the deviceidentifying information and device authentication information in a datastore such as a database.

At block 516, device security server 146 generates and communicates oneor more responses to the registration related requests that have beenreceived. In an example embodiment, the responses identify the deviceidentifier that has been associated with the device 130 that wasspecified in the request. The response may further specify the deviceauthentication information, which may have been included in the originalrequest from the device. In embodiments wherein a certificate signingrequest was made, responsive information including a generatedcertificate is communicated by the device security server 146. Thecertificate may subsequently be used when the device connects toenvironment 110 so that the device can verify that it is connected tothe correct service. In an example embodiment, the device may use thecertificate to perform a TLS exchange with environment 110 to verifythat it is communicating with the intended service. Further, inembodiments wherein device activation codes are associated with devicesand used during subsequent processing, responsive information providedby device security server 146 may comprise a device activation code.

In some embodiments, the response from device security server 146 mayfurther provide environment authentication information that the device130 may use to authenticate that it is connecting to the communicationenvironment 110, as opposed to another system. The additionalinformation provided in the response may comprise, for example, a publickey that corresponds to a private key that is maintained in secrecy bythe device security server 146. When the device 130 subsequentlyattempts to access the environment 110, the device 130 may use thepublic key to decrypt data that is sent by the device communicationsystem 110 so as to authenticate that the device has connected to theintended system.

In embodiments wherein a certificate was requested and received fromdevice security server 146, the manufacturer, or other provider of thedevice, may store the credentials on the corresponding device. Forexample, the manufacturer may store on the device the device identifierassigned by communication system 110, the public key of the privatekey-public key pair associated with the device, and any certificatecreated in response to a certificate signing request. In an exampleembodiment, the provider of the device may use an ImageDevice API callto the device in order to store the information on the device.

In some instances, it may be desirable to associate particular entities,which may be, for example, individuals, users, accounts, organizations,etc., with particular devices and to use this information in connectionwith processing requests to control or communicate with the particulardevice. As discussed in connection with FIG. 5 , a manufacturer ofdevices may register the devices with the device communication system110. The manufacturer may subsequently sell those registered devices tothe public. For example, the manufacturer of devices such asthermostats, washer machines, or any other device, may register thedevices, as described above, and then may sell those devices to thepublic. The purchasers of those devices may then wish to register thatthey had purchased or control particular ones of the devices. In anexample embodiment, the device communication environment 110 may receiveand store information specifying the particular entity (e.g., individualor organization) that is associated with the particular device. Thisassociation between the device and the particular entity (e.g.,individual or organization) may subsequently be used to regulaterequests to control or communicate with the particular device. Inconnection with associating themselves with a particular device, anentity may also activate the device with the environment 110. Where anactivation code was associated with a particular device during themanufacturer's registration process, the purchaser may use theactivation code to activate the device.

FIG. 6 depicts a flow chart of an example process for associating adevice with a particular user, operator, or organization. As shown, atblock 610, device security server 146 receives a request to associateinformation identifying a particular entity (e.g., individual ororganization) with a particular device. The request may be received viaan API provided by device security server 146 and may have originatedfrom any suitable system. In an example scenario, the request mayadditionally comprise a request to activate a device using an activationcode that was specified during a manufacturer device registration. In anexample embodiment, a separate request to activate the device may bereceived. In an example scenario, the request may be received from anexternal service 160 operated by or for the manufacturer of the devicethat receives requests from the purchasers of the devices and formatsappropriate requests to associate the purchaser with the purchaseddevice. In an example scenario, the third party service 160 may receiverequests originating from the purchasers of the device who may input thedevice identifier assigned to the device via any of numerous differentmeans. In an example scenario, the individual may use a mobile deviceapplication that may be provided by or for the manufacturer of thedevice to retrieve the device identifier, and possibly a deviceactivation code, from the device that was purchased. The app may use anysuitable technology to retrieve a device identifier from the device. Forexample, the app may provide for scanning a bar code or QRC code thatwas provided with the particular device and which contains the deviceidentifier. In another embodiment, the mobile application may use nearfield communication (NFC) to retrieve the device identifier from acorresponding transmitter on the purchased device. In yet anotherembodiment, the purchased device may operate as a beacon and operate asan access point with which the mobile application pairs to retrieve thedevice identifier. In still another embodiment, the mobile app may beadapted to receive the device identifier via manual input (e.g.,typing).

The particular entity (e.g., individual or organization) is identifiedin the request received at device security server 146 using any suitableidentifying information. In an example embodiment, the informationidentifying an entity (e.g., individual/organization) may be informationidentifying a particular account with which the particular entity isassociated. For example, information may comprise informationidentifying an account with an existing service such as, for example,the Amazon Cognito service. In an example scenario, the entity thatpurchased the device may enter their account information for such aservice. In an another scenario, the entity may log into the existingservices such as Amazon Cognito in order that their identificationinformation may be retrieved and included in the request. In an examplescenario, the system or service that generates and forwards the requestto device security server 146 may have previously retrieved theidentifying information from the existing service.

In an alternate embodiment, rather than originating at an externalservice 160, the request to pair a device with a particular entity(e.g., individual or organization) may be received directly from amobile application operating on an individual's mobile device withoutthe use of an intervening service or system. In still anotherembodiment, the device itself may have an interface that allows forcollecting entity identifying information such that the device maycommunicate the device identifier and entity identifying information todevice security server 146 without use of a mobile application or even aservice operated by or for the manufacturer.

Referring to FIG. 6 , at block 612, device security server 146 parsesthe received request and identifies the information identifying theparticular entity (e.g., individual/organization). The informationidentifying the entity uniquely identifies the particular entity and mayhave any suitable format.

At block 614, device security server 146 parses the received request andidentifies the device and the corresponding device identifier. Theprocessing may involve authenticating that the request is legitimate forthe particular device identified in the request. For example, therequest may contain additional information that has been signed using aprivate key of the public-private key pair that was associated with theparticular device during the device registration process. Devicesecurity server 146 may decrypt the received data with the public keyand, depending upon the value of the decrypted data, determines whetheror not the request is a legitimate request for device corresponding tothe device identifier. If the processing indicates the request is notlegitimate, processing terminates and a response indicating the such isreturned.

At block 616, assuming device security server 146 determines the requestis, in fact, a legitimate request with respect to the identified device,device security server 146 associates the device identifier with theinformation identifying the particular entity (e.g., individual ororganization) as received in the request. For example, device securityserver 146 may store the device identifier in association with theinformation identifying the particular entity such as, for example, anaccount of the particular entity. Device security server 146 may storethe device identifier in relation to the information identifying aparticular entity in a relational database.

In an embodiment wherein the request to associate a device with anentity additionally comprises a request to activate a device has beenreceived, device security server 146 confirms the activation code thatwas received is consistent with the activation code that was previouslystored for the particular device. Device security server 146 mayassociate the activation code with the entity.

At block 618, device security server 146 generates and communicates aresponse to the request. In an example embodiment, the responseidentifies that the device identifier has been associated with theinformation identifying the entity as specified in the request.

At the conclusion of the processing at block 618, device security server146 will have stored thereon for a particular device, informationspecifying the unique device identifier, authentication information suchas a public key, a public key certificate where one has been requestedand created, information provided by the manufacturer identifying thedevice (e.g., a serial number), and information identifying theparticular entity associated with the device. Where the processinginvolved use of an activation code, device security server 146 may havealso formed a relation between the activation code and the particulardevice. This information may subsequently be used by the communicationenvironment 110 to authenticate communications from the particulardevice, as well as to limit communications to the device to thosecommunications originating from the particular individual ororganization that has been associated with the device.

It will be appreciated that while devices 130 may be activated by theowner or purchaser of the particular device as described in connectionwith FIG. 6 , in some circumstances, the provider of the device, such asthe manufacturer, may activate the device. The processing issubstantially the same, but initiated by the provider of the devicerather than the purchaser. In such a scenario, when a device is receivedby its purchaser, it may have already been activated.

Once a device has been registered with environment 110 and an entity(e.g., individual or organization) has been associated with a device,the device may communicate with environment 110. FIG. 7 depicts a flowchart of example processing performed in connection with a device 130initially requesting to connect to computing environment 110. At block510, gateway server 140 receives a request to connect. The request maybe formatted in a number of different formats and may use any number ofdifferent protocols such as, for example, HTTPS, CoAP, MQTT, etc. In anexample embodiment, the request may comprise a request to start atransport layer security (TLS) session. In such an instance, the TLSsession request may include the public key certificate that wasgenerated during the device registration processing as discussed inconnection with FIG. 5 .

At block 712, gateway server 140 validates or authenticates theparticular device. For example, gateway server 140 may determine whetherthe particular device 130 is one that should be able to access server140 and environment 110. In an example embodiment, the information thatis received and stored during the registration process may be used toauthenticate the device when it attempts to communicate with theenvironment 110. In an example embodiment, the authentication mayinvolve a transport layer security (TLS) handshake. In an examplescenario where a TLS session was initiated by a request from device 130,gateway server 140 validates the certificate. Gateway server 140 maycommunicate with device security server 146 in order to confirm that thereceived certificate matches the certificate that was previously createdduring the device registration processing as described above inconnection with FIG. 5 . In an example embodiment, gateway server 140may issue a request, which may include the device identifier, to thedevice security server 146 and receive a corresponding certificate. Inan alternate embodiment, the request to the device security server 146may simply identify the certificate without specifying a particulardevice identifier, Gateway server 140 may compare the certificatereceived from the device 130 with the certificate received from thedevice security server 146.

The validation or authentication processing may further compriseprocessing an encrypted data item using the public key that was storedin association with the device during registration. For example, device130 may have encrypted a known data item using its private key of thepublic-private key pair that was identified and stored with securityserver 146 during device registration. During the authenticationprocess, gateway server 140 may retrieve the public key from securityserver 146 and use it to decrypt the received encrypted data. Gatewayserver 140 evaluates the unencrypted data to determine if the data was,in fact, encrypted using the private key corresponding to the registeredpublic key and, therefore, received from the actual registered device.

It will be appreciated that in addition to gateway server 140 and devicesecurity server 140 authenticating device 130, device 130 may request acertificate from gateway server 140. In response, gateway server 140and/or device security server 146 may generate and/or retrieve acertificate. In an example embodiment, gateway server 140 and/or devicesecurity server 146 may generate and transmit a certificate byencrypting a known data item using a private key that corresponds to apublic key that may have been exchanged with the device during theregistration process. Device 130 evaluates the certificate to determinewhether it indicates that device 130 is, in fact, communicating with theintended communication environment 110. For example, device 130 maydecrypt the received data using a public key that was provided by devicesecurity server 146 during the registration process. If device 130determines that the decrypted data is acceptable, device 130 candetermine that it is communicating with the appropriate environment.

In an example embodiment, in addition to performing certificateexchanges and/or encrypted data exchanges between a device 130 andenvironment 110, a further internal check may be made prior to allowingfor communication between a particular device 130 and environment 110.For example, device security server 146 may maintain informationspecifying particular devices that have been identified as excluded fromcommunication with environment 110. The particular devices may havepreviously presented security or operational issues that indicated thedevices should not be trusted to communication with environment 110. Theinformation regarding whether devices are excluded may be stored in anymanner that is suitable. In an example embodiment, the information maybe stored by device security server 146 as an extra memory bit inrelation to information about the particular device. Accordingly, devicesecurity server 146, after authenticating a device using certificateinformation, may determine whether the data maintained for theparticular device indicates it should be excluded from communicatingwith environment 110.

In the event that the particular device is not identified as excludedfrom communication with system 110, device security server 146communicates with gateway server 140 to indicate that communication withthe particular device may proceed.

Assuming gateway server 140 determines that the particular device mayconnect, at block 714, gateway server 140 may assign a connectionidentifier for purposes of identifying the particular connection withthe particular device in environment 110. Gateway server 140 may assignany suitable identifier. In an example embodiment, gateway server 140may assign an IPv6 formatted identifier. Gateway server 140 stores anysuch identifying information that may be assigned to the device 130.Information relating to the particular connection such as the socketused for the connection and the protocol type of the request may bestored with the identifying information for the particular device 130.Any information regarding the device, including any device identifier,connection identifier, port, and/or protocol, are stored in relation toany hardware identifying information for the particular device. Forexample, the information may be stored in relation to a UUID or MACaddress of the particular device.

At block 716, the existence of the device 130 and its connection withthe particular gateway server 140 is published so that other gatewayservers 140 may reference the information in the future. In an exampleembodiment, the gateway server 140 to which the particular deviceconnected, requests that distributed hash server 220 store hash valuescorresponding to the device identifier, the connection identifier, andan identifier for the particular gateway server 140 to which the deviceconnected. This information may be used as described below to locate theparticular gateway server responsible for processing requests from theparticular device 130.

At block 718, the particular gateway server 140 may register theparticular device 130 with device security server 146. For example,gateway server 140 may forward any security-related informationregarding the particular device 130 such as any key values that may needto be used during communication with the device. Device security server146 may provide information regarding any limitations such as roles andpolicies that may be imposed when the particular device connects to theenvironment 110.

At block 720, gateway server 140 generates and transmits anacknowledgement of the device having connected with the computingenvironment via gateway server 140.

Gateway sever 140 may be adapted to support asynchronous messages overboth connection-based and connection-less transport protocols between adevice endpoint and an endpoint in environment 110. In an exampleembodiment, gateway server 140 supports message exchange patternsbetween a device endpoint and a cloud endpoint such as, for example,one-way device to cloud messaging; device initiated request/responsemessaging; one-way cloud to device messaging; and cloud initiatedrequest/response messaging. In an example scenario wherein a deviceconnects to gateway server 140 using a transport protocol over apersistent connection such as TCP, gateway server 140 associates thespecific socket that is used to hold the connection. Generally,connection-based communication protocols such as transmission controlprotocol (TCP) first establish a communication session, and then use theestablished session to communicate data. Connection-based communicationprotocols offer reliable delivery of data through sequencing, errorchecking, and flow control. Generally, a session is established througha process of a handshake. One the connection is established, data can bereliably sent. Each datagram that is sent is assigned a sequence number.The sequence numbers ensures that the destination computer canreassemble the data is the proper order as it is received. Thedestination computer will send acknowledgments for segments that arereceived. If the source computer does not receive an acknowledgementwithin a certain amount of time, the segment will be retransmitted thusensuring delivery of information.

The disclosed environment may also employ connection-less communicationprotocols. Connection-less communication protocols such as, for example,user datagram protocol (UDP), do not rely upon an established connectionfor communicating data. When information is sent, acknowledgements arenot returned so it is just assumed that it has been received.Connection-less protocols are more often used in one-to-many situationswhere information is sent using a broadcast or for multicasting (TCP isused for one-to-one communication). In an example scenario wherein adevice connects to gateway server 140 using a connection-less transportsuch as UDP, gateway server 140 associates the remote IP and port pairwith a connection id.

In an example embodiment, whether or not gateway server 140 tracks theconnection state of individual devices depends on the protocol used bythe device to communicate with gateway server 140. For example, whetheror not gateway server 140 tracks connection state depends upon whetherthe protocol is stateful or stateless.

In an example scenario, a request may be formatted as a CoAP request.CoAP, by default, uses UDP as the transport. While UDP is consideredstateless, CoAP requires state to be maintained about a single messageexchange. A message exchange may be defined as a request/responsesequence with state maintained by the system if the confirmable messagetype is used. In an example embodiment, a confirmable CoAP message canbe redelivered until a positive ACK is received. Gateway server 140 maykeep track of redelivery attempts and whether or not a message exchangehas completed. In such an embodiment, message exchange state may beshort lived and may not be required after TTL expires. When CoAP is usedwith UDP, gateway server 140 may not maintain connection state since UDPis connection less.

In another example scenario, requests may be received using the MQTTprotocol. Typically, MQTT requires a server to store client stateinformation in an attempt to reduce per-message communication overhead.MQTT also defaults to using TCP as the transport mechanism. When along-lived TCP connection is used, gateway server 140 maintains theconnection state of a device. In such a scenario, an MQTT client candisconnect and reconnect using the same client identifier. Gatewayserver 140 may maintain MQTT state information per client identifier forQoS level 1 and 2.

SPDY and HTTP/2 support multiplexing data over a single TCP connection.In an embodiment supporting SPDY and HTTP/2, gateway server 140 maymaintain connection state for each device and route messages over theopened TCP connection.

FIG. 8 depicts a flow chart of example processing performed inconnection with receiving a request to perform a function or commandfrom a device 130 at computing environment 110. As shown, at block 810,a request is received from a device 130 into computing environment 110.In an example embodiment, the request is received at load balancer 21.In an example scenario, the request may be to perform an update functionon a data item stored in a network services server 150. In anotherexample scenario, the request may be to perform a particular processingfunction or command on a particular data item stored in a networkservices server 150. In some instances, the request itself may notspecify a particular function that is being requested, but relies uponcomputing environment 110, and, in particular, gateway server 140, toidentify a function corresponding to a request. In an example scenario,the request may specify a value such as a device measurement and relyupon computing environment 110 to identify that the correspondingrequested function is to update a value maintained in the environmentfor the particular device.

In an example embodiment, application programming interfaces (API's) maybe defined for interfacing with gateway servers 140. Different API's maybe defined for different communication protocols. For example, API's maybe defined for HTPP communications by devices 130 for communication withgateway servers 140. In an example scenario, an API callsendMessageToCloud(deviceId, path, payload) may be defined for HTTPcalls by devices 130 to gateway servers 140. This particular call allowsthe calling device 130 to send a message to computing environment 110.The sendMesssageToCloud call may be called with information specifyingvalues for parameters deviceID, path, and payload. The “deviceID”parameter represents the unique identifier for the device 130 sendingthe call. The “path” parameter may be a uniform resource locator (URL)path that conveys message type or any other contextual information ofthe message. In an example embodiment, the “payload” parameter is usedto communicate information about the particular function or command thatis being requested and is used as an abstract interface to communicatemessages to the gateway server 140. The “payload” information maysubsequently be operated on by device registry server 142 in order todetermine what functions or commands should be executed in system 110 inresponse to the request. It will be appreciated that since HTTP is asynchronous request/response protocol, it is possible that a call ofsend MessageToCloud may block until some timeout threshold.

In a further example, an API call longPollMessage( ) may be defined andsent by devices 130. The longPollMessage( ) call may be used by a longpoll client in order to determine if there are new messages that wereinitiated by servers in computing environment 110 for the particulardevice 130. This message also may effectively block until a timeoutthreshold.

Gateway servers 140 may also be programmed to accept CoAP messages. Inan example embodiment, gateway servers 140 act as a CoAP endpoints overboth UDP and TCP. CON message types may be used when interacting with adevice 130 over CoAP. If a gateway server 140 receives a request that isnot of CON type, in an example embodiment, server 140 may return aForbidden message. In an example embodiment, the content formatsupported may be, for example, 50—“application/json,” althoughadditional content formats may also be supported.

In an example, embodiment, a CoAP option “65001—-AMZN-CORRELATION-ID”may be defined and mapped to a correlation ID. In an example embodiment,the following mappings may be made for a CoAP message: path-mapped toCoAP option URI-PATH; correlation ID-mapped to CoAP optionX-AMZN-CORRELATION-ID; payload-mapped to CoAP payload; CoAP contenttype-always set to 50; CoAP code-always set as POST; and CoAPtype-always set as CON.

Gateway servers 140 may still further expose an MQTT endpoint such thatdevices 130 supporting MQTT can communicate. In an example embodiment,gateway servers 140 may implement a subset of handling behavior forCONNECT, PUBLISH, SUBSCRIBE, and DISCONNECT and their correspondingresponses. For SUBSCRIBE, the a valid topic may be/{deviceId}/toDevice.Gateway server 140 may return a SUBACK with failure if an MQTT clientattempts to subscribe to any other topic. With respect to PUBLISH, theonly valid topic may be /{deviceId}/fromDevice. Gateway server 140 mayreturn a PUBACK with failure if the MQTT client attempts to publish toany other topic.

Referring to FIG. 8 and block 812, in an example embodiment, loadbalancer server 210 identifies the particular gateway server 140 a thatshould process the particular request. The load balancer 210 at whichthe request was received may have been provisioned with informationindicating that requests from particular devices should be routed toparticular gateway servers 210. This information may have beenprovisioned onto load balancer 210 when the particular device where therequest originated first accessed environment 110 via the load balancer210. Load balancer server 210 queries this information to identify theparticular gateway server 140 that should process the particularrequest. For purposes of illustration, load balancer server 210 mayidentify that the gateway server 140 a is responsible for processing theparticular request which may have originated from device 130 a.

In an alternate embodiment, load balancer 210 may not have informationsufficient to identify the relevant gateway server 140 a as responsiblefor servicing requests from a particular device. In such an embodiment,load balancer 210 may direct a request to a gateway server 140 basedupon available resources and without regard as to the particular gatewayserver that is responsible for servicing the request. Accordingly, itmay be the case that the gateway server 140 that receives the requestfrom load balancer 210 may not be the gateway server 140 associated withor responsible for processing request from the particular device 130.For example, the request may be received at a gateway server other thanthe one that previously communicated with the device. Accordingly, atblock 812, the gateway server 140 at which the request was receivedidentifies another gateway server to service the request. In such anembodiment, the gateway server 140 may query distributed hash server 220to identify a gateway server 140 that corresponds to the particulardevice 130. In an example scenario, gateway server 140 generates a hashvalue of an identifier, e.g., device_id, for the particular device 130from which the request was received. The hash function is the same asthat used when the device previously accessed the system. In an examplescenario, the hash value is taken of the device identifier associatedwith the particular device. The generated hash value is used to querythe distributed hash server 220 in order to locate a gateway server thatis responsible for responding to the request.

As noted above, in an example embodiment, distributed hash server 220comprises a consistent hash ring wherein hash values of identifiers ofgateway servers 140 and are included in hash ring in addition to hashvalues of the device identifiers and connection identifiers. In such anembodiment, hash server 220 first locates the hash value correspondingto the device identifier on the hash ring. Hash sever 220 then locatesthe hash value of a gateway server that corresponds to the located hasvalue of the device identifier. In an example embodiment, hash server220 locates the hash value of a gateway server that is located inproximity on the ring to the hash value of the particular deviceidentifier. The located hash value of a gateway server corresponds tothe gateway server responsible for the particular device from which therequest was made. In an example embodiment, hash server 220 locates thehash value corresponding to the device on the hash ring and then movesclockwise on the ring until finding a hash corresponding to a gatewayserver. The first gateway server hash located upon moving in a clockwisedirection corresponds to the gateway server responsible for respondingto the request.

Referring to FIG. 8 , at block 814, the request is communicated to theidentified gateway server 140 for processing.

At block 816, the identified gateway server 140 queries for logic,policies, or rules for processing the request. In other words, gatewayserver 140 determines what processing is needed in order to respond tothe request. In an example embodiment, the identified gateway server 140is programmed to receive and process requests that include payloadinformation specifying a particular function that is requested to beimplemented. The identified gateway server 140 may also be programmed toreceive and process requests that do not explicitly specify a particularfunction. In an example scenario, a request may be received from adevice such as, for example, a thermostat, that includes informationidentifying the device, e.g., a device identifier, and payloadinformation identifying a temperature reading value from the thermostat.The identified gateway server 140 may be programmed to use theinformation that is specified in the request in order to identify acorresponding function. In an example embodiment, the identified gatewayserver 140 may be programmed to query a data store for a particularfunction that corresponds to information that is provided in the requestfrom the device. The data store may comprise information specifying forparticular devices and particular information or data items that may bereceived from the particular device, a corresponding function that isintended. In an example scenario, the data store may compriseinformation that specifies for a request received from a particulardevice identifier that includes a payload with a temperature reading, astore function should be interpreted to have been requested. In anexample scenario, the identified gateway server 140 may query the datastore, which may be comprised on gateway server, for a particularfunction that corresponds both to the requesting device, which may bespecified by a device identifier, and a data item such as an instrumentreading value included in the request payload. Accordingly, in scenarioswhere the request does not explicitly specify a requested function, theidentified gateway server 140 may be programmed to determine a requestedfunction for a request from a device, based at least in part oninformation such as a device identifier and payload information from therequest.

It will be appreciated that a gateway server 140 may need to processrequests from a wide assortment of devices 130. For example, devices 130may be smart phones, pressure sensors, light switches, or any other typeof device. The various devices 130 may have different functions orcommands that are particular to the particular device or type of device.For example, a device 130 that is a smart phone may communicate commandsto gateway server 140 relating to a current location and/or requests forlocations of other smart phones. Meanwhile, a sensor device located in amanufacturing facility may communicate messages providing updatesrelating to pressure readings. Further, the devices 130 may usedifferent protocols when communicating with environment 110 and, inparticular, gateway server 140. For example, some devices 130 mayforward HTTPs formatted commands. Other devices 130 may forward messagequeue telemetry transport (MQTT) formatted messages. Still other devicesmay transmit constrained application protocol (CoAP) formatted messages.

Because there are so many variations in the types and formats of requestreceived, the identified gateway server 140 determines for theparticular request from the particular device 130 what responsiveactions should be taken. As part of this processing, gateway server 140converts the requests/messages that it receives from devices 130 into aformat that may be used by systems in computing environment 110. Forexample, for a message received from a device, gateway server 140 maygenerate a JSON document the contents of which represent the values forparameters in the call received from a device 130. An example JSONdocument may be as follows:

{  {“header”: {   “deviceId” : “2323-3542-2213543”,   “path” :“/shadow/state”,   “correlationId” : “123-435-423454”   }  },  {“body”:{   // whatever was passed in from developer as the payload.  }  }, {“diagnostic” : {   “connectionId” : “3433-23454-134543”,  “receivedTS” : “343254363446”, //in UTC   “some more diag” : “TBA”   } } }In such an embodiment, values for document elements such as, forexample, deviceID, path, correlationID, and body may be taken from theAPI call that was received by the gateway server 140. The value for“body” element may correspond to a payload provided by the API call andmay be used to determine the specific function within computingenvironment 110 that is intended to be performed.

In an example embodiment, the identified gateway server 140 queries forprocessing actions or functions that should be taken in response to theparticular request for a function from the particular device. In anexample embodiment, device registry server 142 comprises data specifyinglogic or rules for determining processing functions or commands thatshould be executed in response to requests for particular functions orcommands from particular devices or devices having particularcharacteristics. In such an embodiment, at block 614, the identifiedgateway server 140 communicates a request to device registry server 142.The request may specify the function or command received from the device130 along with any related data including the particular device fromwhich the request was received. In an example embodiment, the request isformatted using JavaScript object notation (JSON) and includes thecommand or function that was received from the device 130 along with anydata corresponding to the request. In a particular embodiment, therequest may be formatted as in the example of the preceding paragraph.In such an embodiment, the command or request may be specified in the“body” portion of the JSON document.

Device registry server 142 queries its records of rules to identifyfunctions or commands that should be executed in response to the requestfor the particular functions or commands from the particular device 130.For example, device registry server 142 may determine that theparticular request received from the particular device 130 correspondsto a request to perform a storing function on data in network servicesserver 150. In other words, device registry server 142 may determinethat the particular function received from the particular device 130corresponds to a services function performed by a services server 150.In an alternative scenario, device registry server 142 may determinethat the particular request from the particular device 130 correspondsto a request to perform predefined processing function or command atnetwork services server 150. It will be appreciated that the data andlogic of device registry server 142 allow for differentiating functionsand commands based upon the particular commands and the particulardevice from which the commands were received.

Device registry server 142 formats and transmits a response to thegateway server 140. The response specifies the particular servicesfunctions or commands that should be executed in response to the requestfrom the particular device 130. In an example scenario, the response mayspecify to request that one or more services functions or commands beexecuted that stores a particular data item in a particular networkservices server 150. In an alternative scenario, the response mayspecify that a particular function or command be called that performs aparticular processing operation on a particular data item at aparticular network services server 150. In an example embodiment, theresponse is formatted as a JSON document.

At block 818, the identified gateway server 140 identifies the actionsthat should be taken in response to the request. In an exampleembodiment, the gateway server 140 receives the response from deviceregistry server 142 specifying the processing functions or commands thatshould be executed in response to the request from device 130. At block818, the identified gateway server 140 may parse a JSON documentreturned by device registry server 142.

At block 820, the identified gateway server 140 generates and transmitsa request for processing consistent with the responsive informationreceived from device registry server 142. In an example embodiment,identified gateway server 140 generates and transmits a request to thedispatcher server 230 to implement one or more services functions orcommands identified in the response received from device registry server142. The request generated by the identified gateway server 140 may beformatted as a JSON document or contain JSON formatted text.

In response to the request, dispatcher server 230 generates and formatsone or more requests to other of servers in environment 110. Forexample, when dictated by the request from gateway server 140,dispatcher server 230 formats and generates requests to network servicesserver 150 to execute one or more functions. In an example scenario, therequest may specify performing a function or command to store a dataitem at network services server 150. In other words, the request mayspecify performing a function or command to perform a services functionat services server 150. In another example scenario, the request mayspecify that network services server 150 perform a particular processingfunction or command on a particular data item. When the processing iscompleted, dispatcher server 230 receives a response, including anyresulting data from network services server 150. Dispatcher server 230communicates a response, including any responsive data, to gateway sever140.

At block 822, the gateway sever 140 receives the response. In an examplescenario, the response may comprise data and/or an acknowledgement thatone or more functions were executed.

At block 824, the identified gateway server 140 generates and transmitsa response to the particular device 130 that generated the initialrequest. In an example embodiment, gateway server 140 may communicate arequest to device registry 142 for information relating to formatting ofan appropriate reply to the particular device 130. The request mayspecify the particular responsive information received by gateway server140 as well as identify the particular device from which the originalrequest was received. Device registry server 142 uses its rules andlogic to identify for the particular device an appropriate format forproviding the responsive information. For example, device registryserver 142 may determine the particular function or command that shouldbe requested as well as the format of the particular command. Uponreceiving any information needed for formatting the response, gatewaysever 140 communicates the response to the device 130 from which theoriginal request was received.

Accordingly, gateway server 140 operates to receive requests to performparticular functions or commands, converts those functions or commandsto appropriate functions for execution within the computing environment110, and returns results in a format accepted by the requesting devices130.

According to another aspect of disclosed embodiments, gateway server 140operates to route data and commands originating from within theenvironment 110 or from external servers 160 to devices 130. In theinstance where the request originates from within environment 110, therequest can generally be assumed to be valid. However, for requests thatoriginate from outside of environment, such as from an external server160, computing environment 110 is adapted to validate that the requestwas made by an entity (e.g., individual or organization) that isauthorized to make the request.

FIG. 9 depicts a flow chart of example processing of requests fromenvironment 110 directed to a device 130. In the depicted processing,optional processing at block 906 applies to requests received fromexternal to environment 110 which may require validation of the request.In an example scenario, the owner or operator of a particular device maywish to have the particular device perform a particular action or toassume a particular state. For example, where the device is adishwasher, the owner of the dishwasher may wish to have the dishwasherrun a clean cycle. In an example embodiment, the owner of the device,who previously registered as the entity (e.g., individual ororganization) associated with the device, may use an application such asan application on a mobile device to request that the particular deviceperform a particular operation. The request may be communicated by athird party service 160 maintained by or for the manufacturer of thedevice, which communicates the request to environment 110. In analternative embodiment, the mobile application may directly communicatewith environment 110.

In an example embodiment, a request from an external system 160 may bereceived at device security server 146. In an example embodiment, therequest comprises information identifying the particular entity (e.g.,individual or organization) from which the request originated, thedevice identifier for the particular device that the user is interestedin communicating with, and the requested operation that is beingrequested, which may be formatted as a payload to the request. In anexample scenario, the requested control or operation may specify thatthe identified device assume a desired state.

Device security server 146 processes the request to confirm that theentity (e.g., individual or organization) from which the request wasreceived, is, in fact, an entity designated to control the particulardevice. Device security server 146 compares the entity (e.g., individualor organization) identification information specified in the requestwith the information identifying an entity that was captured and storedby device security server 146 during the registration process asdescribed above. More particularly, device security server 146 validatesthat the information previously stored for the device identified in therequest indicates the particular entity (e.g., individual ororganization) specified in the request is authorized to communicate withthe device. If device security server 146 determines that the user isnot authorized, processing of the request terminates and a correspondingcommunication is generated and transmitted to the system from which therequest originated.

Assuming the entity (e.g., individual or organization) specified in therequest is determined to be authorized to communicate with or controlthe particular device, at optional block 908, the request is processedat device shadow server 144, which maintains state information for eachof the devices registered with the system. In one example embodiment,device shadow server 144 updates the state information for the deviceconsistent with the request. For example, device shadow server 144 mayupdate the desired state for a device to match the state specified inthe request. In an example scenario wherein the request is to change theoperating state of a dishwasher device to indicate the dishwasher is torun a clean cycle, the desired state is changed to reflect the newstate. Device shadow server 144 may forward a corresponding request toimplement the desired change of state.

As shown, at block 910, gateway server 140 receives a request or messagefrom within environment 110 that requires communicating with a device130. The request or message may specify, for example, a particulardevice identifier corresponding to a particular device, or may specify aparticular connection identifier, which corresponds to a connection witha particular device. The request or message may further include apayload, which may be, for example, an action such as a function orcommand that is requested as well as any data corresponding to therequest. In an example scenario, device shadow server 144 may transmit arequest to gateway server 140 to change the operating or processingstatus or state of a particular device 130 or particular connection. Therequest may specify a particular state, actual and/or desired, that aparticular device or connection should assume. Device shadow server 144may issue a request to change status or state in response, for example,to receiving an indication from an external server 160 or from anapplication such as network services server 150 to change the devicestatus or state. In an alternative scenario, gateway server 140 mayreceive a request from dispatcher server 230 to send a message to thedevice. Dispatcher server 230 may issue the request in response, forexample, to a request from a network services server 150. It will beappreciated that servers within environment 110 may use API calls tocommunicate requests to gateway server 140. In an example embodiment,HTTP API calls may be used by applications to request that messages besent to particular devices. An example API call may be, for example,sendMessageToDevice(connectionId, path, payload) which when calledcauses gateway server 140 to send an async message to the device 130identified by the “connectionId” parameter. The “path” parameter may be,for example, a URL path that is passed through to the device and whichidentifies message type or any other contextual information of themessage. In an example embodiment, the “payload” parameter is meant tohold information specifying instructions for performance by theparticular device 130 which ultimately receives the instruction. In anexample embodiment, the “payload” parameter may be a JSON formatteddocument. In such an instance, the JSON document represents the abstractinterface for an application/service to send messages to a device 130.It will be appreciated that the implementation details may differ basedon the communication protocol used by a particular device 130. Anotherexample API call may be sendResponseToDevice(connectionId,correlationId, path, payload) which when called causes gateway server140 to send an async response to an earlier message. The format of thisparticular call is similar to sendMessageToDevice with the exceptionthat sendMessageToDevice( ) also includes a parameter “correlationId”which may be used to correlate the response to a previous request.

At block 912, the gateway server 140 that received the requestdetermines which gateway server 140 of the several that may exist shouldhandle the request. As noted in the discussion of FIG. 2 , there may bemany gateway servers 140. It may be the case that the gateway server 140that receives the request may not be the gateway server 140 associatedwith or responsible for processing request for the relevant device 130.Accordingly, at block 712, the gateway server 140 at which the requestwas received identifies a gateway server to service the request. In anexample embodiment, the gateway server 140 queries distributed hashserver 220 to identify a gateway server 140 that corresponds to aparticular device or, in some instances, corresponds to the particularconnection that is identified in the request. In an example scenario,gateway server 140 generates a hash value of an identifier which may be,for example, a connection identifier for a particular device or a deviceidentifier. The hash function that is used to generate the hash value isthe same as that used when the device previously accessed the system togenerate a connection. The generated hash value is used to query thedistributed hash server 220 to locate a gateway server that isresponsible for responding to the request.

In an example embodiment, distributed hash server 220 comprises aconsistent hash ring wherein identifiers of gateway servers 140 areincluded in hash ring in addition to hash values of the deviceidentifiers and connection identifiers. In such an embodiment, hashserver 220 first locates the hash corresponding to the connectionidentifier on the hash ring. Hash sever 220 then locates the hash of agateway server that is located in proximity on the ring. The locatedhash of a gateway server corresponds to the gateway server responsiblefor the particular connection with the desired device. In an exampleembodiment, hash server 220 locates the hash corresponding to the deviceon the hash ring and then moves clockwise on the ring until finding ahash corresponding to a gateway server. The first gateway server hashlocated upon moving in a clockwise direction corresponds to the gatewayserver responsible for responding to the request.

Upon identifying a particular gateway server, at block 914, the requestis communicated to the identified gateway server 140 for processing. Inan example embodiment, the request comprises information specifying theconnection identifier for communicating with a particular device.

At block 916, the identified gateway server 140 queries for or performsprocessing to identify a function or command corresponding to thatspecified in the original request. In an example embodiment, identifiedgateway server 140 may request that device registry server 142 formatthe request for the particular connection with the particular device.Device registry server 142 may comprise rules or logic that indicatesfor particular functions or commands that may be issued by servers inenvironment 110, corresponding functions, commands, data, and formattingfor requests to be sent to particular devices. Accordingly, at block916, device registry server 142 may query to identify a function orcommand compatible with the identified connection and corresponding to afunction or command specified in the original request. In an examplescenario wherein the original request received from shadow server 144indicates to change an actual and/or desired processing state or status,device registry server 142 identifies for the particular device andconnection, the appropriate function or command and the appropriateprotocol formatting.

At block 918, the identified gateway server 140 identifies the actionsthat should be taken in response to the request. In an exampleembodiment, the gateway server 140 receives the response from deviceregistry server 142 specifying the processing functions or commands thatshould be executed in response to the request from shadow server 144. Atblock 918, the identified gateway server 140 may parse a JSON documentreturned by device registry server 142.

At block 920, the identified gateway server 140 communicates to thedesired device 130 the identified request including any function orcommand appropriate for the device and corresponding to that in theoriginal request. The request is formatted as needed for the particularconnection to the particular device. In an example scenario, the requestmay be formatted as CoAP request and specify a message indicating tochange status. In another scenario, the request may be formatted as aMQTT publish request and specify a new value for use by the device. Instill another scenario, the request may be formatted as an HTTPs requestand specify a message indicating to change status of an operatingparameter.

It will be appreciated that the identified gateway server 1440communicates with the desired device 130 consistent with the connectiontype maintained for the particular device. For example, where theconnection with a device is a connection-based connection that employs aconnection-based communication protocol such as TCP, gateway server 140transmits the request or message over the connection-based connectionusing the connection-based protocol upon identifying or generating theparticular request or message. In the scenario wherein the connection isa connection-less connection that employs a connection-lesscommunication protocol such as UDP, and the particular device is notcurrently connected, gateway server 140 may wait for the particulardevice to connect with the gateway server 140 before transmitting therequest or message and may use the connection-less communicationprotocol.

At block 922, the identified gateway server 140 receives acknowledgmentof the request. In an example scenario, the identified gateway server140 may receive a CoAP acknowledgement. In another scenario, theidentified gateway server 140 may receive a MQTT publish back message.

At block 924, the identified gateway server 140 communicates anacknowledgment to the particular device that requested that the messagebe sent. For example, the identified gateway server 140 may communicatean acknowledgement to device registry server 142 or dispatcher server230 as appropriate.

It will be appreciated that during the operation of the of theenvironment 110, useful data is collected regarding the devices andtheir operation. For example, as the devices change states, deviceshadowing server 144 captures the various states, the durations thedevices were in the various states, the requests that resulted in thechange in states, etc. Likewise, as requests to perform functions arereceived at gateway server 140 from devices, the particular requests,the functions performed in response to the request, and the timing ofthe requests are captured by the gateway server 140 and the deviceregistry server 142. Services systems 150 that provide services inresponse to device requests likewise collect data about the requests,the functions performed, etc.

The data that is collected over time by the systems comprised inenvironment 110 may serve as a useful resource by which to evaluate theoperation of the devices. The collected data reflects operations acrossmultiple devices and device types and may provide valuable insights thatmay be used to improve operation of devices.

In an example embodiment, a server, which may be, for example, aservices server 150, may use the data that has been collected over timeto evaluate future data or commands to be sent to the one or moredevices 130. FIG. 10 depicts a flow chart of an example process forevaluating collected data. As shown, at block 1010, a services server150 may aggregate relevant data collected over time relating to devices130. For example, services server 150 may aggregate data collected bydevice shadow server 144 relating to device states including, forexample, the various states entered by the various devices, the durationthe devices were in the states, the requests that resulted in the statetransitions, and any other relevant information. Services server 150 maycollect and aggregate data from gateway server 140 and device registryserver 142 regarding the requests to perform functions that werereceived, the functions that were performed in response to the requests,the timing of the requests, etc. Services server 150 may also interfacewith other services servers that may have been receiving and collectingdata from various of the devices over time. For example, a particularservices server 150 may have been receiving via the communicationenvironment 110 data relating to various traffic monitoring devices suchas cameras and motion detectors. At block 1010, the data collected fromthe traffic monitoring devices may be aggregated for evaluation.

At block 1012, the services server 150 processes the aggregated data.The processing may involve determining whether the aggregated dataindicate particular commands or data should be communicated toparticular devices. For example, state information aggregated for aseries of light devices may indicate that the lights have consistentlybeen in an “on” state during hours when the building in which the lightsare situated is closed. In such an instance, services server 150 maydetermine to send commands to the light devices to switch to an “off”state during particular hours of the day. In another example scenario,data aggregated from another services server for various trafficmonitoring devices (e.g., motion detectors, cameras, etc.) may indicatethat traffic in the area of the devices increases at a particular timeon particular days of the week. In such an instance, services server 150that aggregated the data may determine to send commands to trafficsignal devices (e.g., stoplights) in the area to change the duration andpattern of traffic light changes (e.g., how long a light is maintainedas green).

Referring to FIG. 10 , at block 1014, the services server 150 thataggregated and processed the data, communicates commands and/or dataconsistent with the determination made by the server based upon theaggregated data. For example, in the scenario wherein the processingidentified several light devices that were on during hours when abuilding is in use, the services server 150 may communicate commands tothe light devices to switch to an off state during particular hours ofthe day. In the scenario wherein the processing identified from datacollected at traffic monitoring devices that traffic in the area of thedevices increases at a particular time on particular days of the week,the services server 150 may communicate commands, via gateway 140, tothe traffic signal devices (e.g., stoplights) in the area to change theduration and pattern of traffic light changes (e.g., how long a light ismaintained as green).

It will be appreciated that the types of data that is aggregated and thecorresponding processing of that data may vary greatly. Likewise, thecommands that are taken in response to the processing of the data mayvary greatly depending upon the types of devices and the actions thatthey perform. In all events, however, the device communication system110 allows for the aggregation of data, processing of the data, andcommunication of responsive commands to devices based upon theevaluation of the data.

Example Device State Management Features

FIG. 11 depicts an example computer networking architecture for devicestate management. The architecture of FIG. 11 is identical to thearchitecture previously depicted in FIG. 1 , with the exception thatFIG. 11 explicitly shows a group of cached device representations1130A-N within device shadow 1100. In particular, each devicerepresentation 1130A-N corresponds to a respective connected device130A-N and includes cached information for the respective connecteddevice 130A-N. As will be described in greater detail below, each cacheddevice representation 1130A-N may include one or more device staterepresentations for the respective device 130A-N. In some examples, thedevice state representations may indicate various device states. Forexample, for a light, the device state representations may indicatepower states, such as Power=On or Power=Off, and/or color states such asColor=Red, Color=Yellow, Color=Green. As another example, for a sensor,the device state representations may indicate particular pressurereadings or ranges of pressure readings.

Also, in some examples, the device state representations may includeboth actual device state representations and desired device staterepresentations. For example, when a device reports that it has assumeda first state, an actual device state representation for the device maybe updated to indicate that the device has assumed the first state. Arequest may subsequently be received from an application for the deviceto assume a second state. In response, a desired device staterepresentation for the device may be updated to indicate that a requesthas been issued for the device to assume the second state. The devicemay then be instructed to assume the second state, and the device maythen report that it has assumed the second state. The actual devicestate representation may then be updated to indicate that the device hasassumed the second state, and the desired device state representationmay then be cleared such that it ceases to indicate the second state.

FIG. 12 depicts an example system for requesting a state change to adevice. In particular, as shown in FIG. 12 , application 1210 mayinteract with a state management application programming interface (API)1215 to issue various requests with respect to a device 1240. In theparticular example of FIG. 12 , the device 1240 may be a light withcolor states such as Color=Red, Color=Yellow, and Color=Green. The statemanagement API 1215 may interact with a state data store 1220 to processrequests from the application 1210, such as by writing information tothe state data store 1220 and/or reading information from the state datastore 1220. The state data store 1220 may include device staterepresentations associated with various connected devices, such asdevice 1240. For example, state data store 1220 may store device staterepresentations 1205 corresponding to device 1240. Device staterepresentations 1205A-C are shown in FIGS. 12 and 13 to depict thechanging values of device state representations 1205 at variousdifferent state update stages. In particular, as shown in FIG. 12 ,device state representations 1205A indicate the values of device staterepresentations 1205 just prior to application 1210 issuing a stateupdate request 1251, which will be described below. Specifically, devicestate representations 1205A show that an actual state of device 1240 isColor=Yellow and that a desired state of device 1240 is empty. Theactual state value (i.e., Color=Yellow) indicates that Color=Yellow isthe last state reported by device 1240. The desired state value (i.e.,empty) indicates that no request to change the state of device 1240 iscurrently being processed. Application 1210 may for example, obtain anindication of the device state representations 1205A by issuing one ormore state read requests, which are described in greater detail below.

In the particular example of FIG. 12 , application 1210 may issue arequest 1251 for the device 1240 to update its state to Color=Red (i.e.,to assume the color red). Application 1210 may issue the request 1251 tostate management API 1215. The request may, for example, identify thedevice 1240 using a device identifier, which, as set forth above, may bea unique identifier assigned to the device. Upon receiving the request1251, state management API 1215 may begin to process the request 1251 byissuing an update instruction 1252 for state data store 1220 to updatethe desired state representation for the device 1240. In particular,because the request 1251 is for the device 1240 to assume the Color=Redstate, the request 1251 will cause the state data store 1220 to changethe desired state representation for the device 1240 from empty toColor=Red. This change is reflected in device state representations1205B, which show the device state representations 1205 after thedesired state representation is updated from empty to Color=Red inresponse to request 1251. It is noted that, in device staterepresentations 1205B, the actual device state representation remains asColor=Yellow. This is because device 1240 has not yet changed its actualstate to Color=Red or reported any such change to its actual state.

It is noted that, similar to request 1251, the desired staterepresentation update instructions 1252 may also identify the device1240 by its corresponding device identifier. Upon receiving updateinstructions 1252, the state data store 1220 may use the deviceidentifier to access and edit the device state representations 1205 forthe device 1240. In some examples, state data store 1220 may be a keyvalue data store, which may use information, such as device identifiers,as a key value by which device state representations may be organized,accessed, and edited.

Upon updating the desired state representation (as shown in staterepresentations 1205B), the state data store may generate, or cause tobe generated, an update event 1253 indicating that the staterepresentations 1205 for the device 1240 have been updated. In someexamples, the event triggered processing function 925 may be acloud-based function, while, in some other examples, the event-triggeredprocessing function 925 may be a local function that is local to statedatastore 920, state management API 915, and/or other associatedcomponents. The update event 1253 may trigger an event-triggeredprocessing function 1225 to execute various instructions for furtherprocessing of the state representation update. The event-triggeredprocessing function 1225 may, for example, obtain the device identifierfor device 1240 as well as information regarding the actual staterepresentation value for device 1240 (i.e., Color=Yellow) and thedesired state representation value for device 1240 (i.e., Color=Red). Insome examples, the event-triggered processing function 1225 may thencompare the actual and desired state representation values in order todetermine that the desired state differs from the actual state. Upondetermining that the desired state differs from the actual state, theevent-triggered processing function 1225 may provide state changeparameters 1254 for the device 1240 to gateway 1230. The state changeparameters 1254 may, for example, indicate the device identifier fordevice 1240, the desired state (i.e., Color=Red), and the actual state(i.e., Color=Yellow).

It is noted that the event-triggered processing function 1225 is merelyone non-limiting example of how an update to a desired staterepresentation may be identified and processed to generate state changeparameters 1254. In some examples, the event-triggered processingfunction 1225 may consume computing resources only at times when it istriggered and invoked in response to a state representation updateevent. Accordingly, event-triggered processing function 1225 may, insome examples, be more efficient than other processing functions orcomponents that execute continuously or that regularly consume computingresources. By reducing the use of computing resources, event-triggeredprocessing function 1225 may reduce costs and improve efficiency withrespect to processing of state change requests. It is noted however,that the disclosed techniques are not limited to the use of anevent-triggered processing function 1225 for processing of state changerequests and that any other appropriate processing functions orcomponents may be employed in addition, or as an alternative to,event-triggered processing function 1225. Also, in some examples, statedata store 1220 may be capable of directly contacting or notifyinggateway 1230 of state change updates, including changes to the desiredstate.

Upon receiving the state change parameters 1254, gateway 1230 may issuea state change command 1255 to the device 1240. In some examples,gateway 1230 may generate the state change command 1255 by convertingthe information within the state change parameters 1254 into a formatthat the device 1240 can understand. For example, the state changeparameters 1254 may sometimes be provided as a JSON (JavaScript ObjectNotation) formatted message, and the gateway 1230 may convert the JSONmessage into a binary or other format that can be processed by thedevice 1240. Upon receiving the state change command 1255, the device1240 may change its state to the desired state indicated by the command,which, in the example of FIG. 12 , is the Color=Red state.

When the device 1240 changes its state its state to Color=Red, thedevice 1240 may report an indication of the state change, which may, inturn, cause the actual state representation for the device 1240 to bechanged to Color=Red. Referring now to FIG. 13 , an example system forreceiving and processing a state change indication from a device willnow be described in detail. In particular, as shown in FIG. 13 , oncethe device 1240 has changed its state from Color=Yellow to Color=Red,the device 1240 may transmit an indication 1310 of the state change toColor=Red. The indication 1310 may be received by gateway 1230, whichmay, based on the indication 1310, generate a request 1311 to statemanagement API 1215 to update the actual state representation for thedevice 1240. The gateway 1230 may, in some examples, generate therequest 1311, at least in part, by converting information from a formatof the indication 1310 into to a different format. For example, theindication 1310 may sometimes include information transmitted from thedevice 1240 in a binary format, and the request 1311 may sometimesinclude information converted to a JSON format. The request 1311 mayinclude, for example, the device identifier for device 1240 as well asan indication of the change of the actual device state to Color=Red.

Upon receiving the request 1311, the state management API 1215 mayperform, within state data store 1215, an update 1315 of the actualstate representation for the device 1240 from Color=Yellow to Color=Red.Device state representations 1205C indicate the values of device staterepresentations 1205 after the performance of update 1315. Inparticular, as shown by representations 1205, the actual staterepresentation is changed to Color=Red.

When a light changes colors, the light may often change immediately ornearly-immediately from a current state to a desired state. However, insome examples, it may not be possible and/or desirable for a device toimmediately change from a current state to a desired state. In oneparticular example, it may be possible to request a desired speed statefor a vehicle motor. In this example, when a large change in speed isrequested (e.g., a change from 0 revolutions per minute (rpm) to 100rpm), an immediate change from a current state to a desired state couldresult in motor burn or other motor damage. Thus, it may sometimes bedesirable for a device to assume various intermediate states before ittransitions to a desired state. Accordingly, in these and other cases,additional desired and/or reported states may sometimes be employed. Inparticular, in some examples, some additional states may be used toindicate characteristics based, at least in part, on which a device willtransition in association with another state and its desired value. Someadditional states may be used to designate values for characteristicssuch as trajectories, gradients, accelerations, rates of change, andother characteristics. In some examples, some additional states may beset to various specific values, maximum, minimum, or average values, ormay indicate mathematical or other relationships, such as linearrelationships or more complex mathematical relationships. As will bedescribed in the example below, a desired value for an additional statemay be received from an application and then indicated to the device inorder to control the transitioning of the device.

For example, in the case of a motor, an application may request twodesired states for the motor: a MotorDesiredRate and a StartProfile. TheStartProfile may be used to indicate how the motor will transition froma current speed to the desired speed of the MotorDesiredRate. In oneparticular example, these two desired states may be set as shown below:

-   -   MotorDesiredRate=100 rpm    -   StartProfile=10 milliamperes per second (ma/s)

The device may then respond with the following reported actual states:

-   -   MotorDesiredRate=100 rpm    -   StartProfile=10 ma/s    -   CurrentRate=0 rpm

The above reporting from the device indicates that the actual state forthe MotorDesiredRate has been set to 100 rpm, which matches the desiredstate for the MotorDesiredRate of 100 rpm. The above reporting from thedevice also indicates that the actual state for the StartProfile hasbeen set to 10 ma/s, which matches the desired state for theStartProfile of 10 ma/s. Additionally, the reported CurrentRate stateindicates that the motor's actual current rate is 0 rpm. Thus, eventhough the motor has not yet spun up, the device's reported actual statefor both MotorDesiredRate and StartProfile match the desired states. Thedevice may then send updates of its CurrentRate to inform the servicehow fast the motor is currently running as it transitions from 0 rpm to100 rpm.

As set forth above, in addition to requesting a state change (e.g., byissuing a write request) to the device state representations,applications may also request to obtain states (e.g., by issuing a readrequest) from the device state representations. FIG. 14 is a diagramdepicting example actual state and desired state read requests. Inparticular, as shown in FIG. 14 , state data store 1220 includes devicestate representations 1205B for device 1240, which include an actualstate representation that indicates the Color=Red state and a desiredstate representation that indicates the Color=Yellow. As also shown inFIG. 14 , applications 1410 and 1415 may each issue read requests toread from device state representations 1205B. In particular, application1410 may issue a read request 1420 to read the actual state of device1240. The actual state read request 1420 may be received and processedby state management API 1215, which may read the actual staterepresentation of Color=Yellow from the state representations 1205Bwithin state data store 1220. A response 1421 indicating the actualstate representation value of Color=Yellow may then be returned toapplication 1410. Additionally, application 1415 may issue a readrequest 1422 to read the desired state of device 1240. The desired stateread request 1422 may be received and processed by state management API1215, which may read the desired state representation of Color=Red fromthe state representations 1205 within state data store 1220. A response1423 indicating the desired state representation value of Color=Red maythen be returned to application 1415.

It is noted that, in the example of FIG. 14 , both actual state readrequest 1420 and desired state read request 1422 may be processedwithout directly engaging the device 1240. Thus, it can be seen thatstate data store 1220 and the device representations stored therein mayallow applications to obtain the actual and/or desired states ofrequested devices at any time without the need to contact the requesteddevices in response to each state read request. This may, for example,reduce the amount of communications usage of other computing resources,as well as reducing the time and expense required to obtain and provideindications of device states to requesting parties. Additionally, asnoted above, due to limited power supplies and other limitations,connected devices may sometimes not always be reachable using networkcommunications, and the use of stored device state representations mayallow indications of device states to be obtained and provided, evenwhen the devices themselves may be temporarily unreachable. It isfurther noted that, in some examples, in addition to current states,historical states may also be stored within state data store 920 andprovided upon request. For example, a listing of device state over oneor more time periods may be requested and provided to a requestingapplication, without the need to directly engage the underlying physicaldevice.

In some examples, relationships between a physical device and its deviceshadow, including, for example, its respective device representations,may be configurable by, for examples, developers and other parties. Forexample, in some cases, rules may be set that may define under whichconditions and/or in response to which API calls a physical device maybe directly engaged. For example, an amount of time may be specifiedthat indicates a threshold time period beyond which a cached staterepresentation may considered be unreliable or may otherwise triggerdirect engagement of a physical device. To illustrate this concept,consider the scenario in which a particular sensor reading from a devicemay be reported by a device and saved in its respective device staterepresentation. Now suppose that the sensor reading from the device maybe considered out-of-date or unreliable after a time period of threehours. In this scenario, a three hour threshold time period maysometimes be defined after which it is required for the device to bedirectly engaged in response to state read requests. For example, when aread request is issued, by an application, to obtain the state of thedevice, it may be determined how long it has been since the device'sstate representation was last updated. If the device's staterepresentation has been updated within the past three hours, then thedevice state indicated by the device's stored state representation maybe returned to the application without directly engaging or contactingthe device. By contrast, if the device's state representation has notbeen updated within the past three hours, then the device may becontacted and instructed to report its current state. Upon receiving thereported current state, the device's state representation may be updatedto indicate the current state and an indication of the current state mayalso be reported back to the requesting application. In some examples,in order to allow an easy and efficient determination of a time periodsince a device state representation was last updated, a timestamp orother indication of an update time of a device state representation maybe stored within the device state representation or at an accessiblelocation.

Referring now to FIGS. 15A-B, some example processes for managing devicestates will now be described in detail. The processes of FIGS. 15A-Bmay, in some examples, be performed by a device management service thatmay, for example, manage and report device states, enable control ofdevices, and perform other operations. The connected devices may connectto the service intermittently over one or more networks. As shown inFIG. 15A, at operation 1510, an indication may be received associatedwith a device assuming a first state. As set forth above, in oneparticular example, the device may be light that has assumed aparticular color state, such as a Color=Yellow state. As also set forthabove, the indication may be reported by a device, such as device 1240of FIGS. 12 and 13 , to a gateway, such as gateway 1230 of FIGS. 12 and13 . In some examples, the indication received at operation 1510 mayinclude data provided by the device using a binary format.

At operation 1512, an actual device state representation associated withthe device may be updated to indicate the first state. As set forthabove, the device management service may maintain a number of devicestate representations associated with the connected devices and performoperations based, at least in part, on the device state representations.The device state representations may include, for example, actual devicestate representations and desired device state representations thatindicate actual and desired device states, respectively, for theconnected devices. The device state representations may be stored in oneor more key value data stores separate from and/or external to theconnected devices. In some examples, operation 1512 may includeconverting, by the device gateway, information within the indicationreceived at operation 1510 into a different format that may be employedby service components, such as a JSON format. Operation 1512 may alsoinclude providing the converted information, which may include a deviceidentifier for the device and an indication of the assumption of thefirst state, to a state management API, such as state management API1215 of FIGS. 12 and 13 , which may, in turn, update a state data storethat includes the device state representations, such as state data store1220 of FIGS. 12 and 13 . As set forth above, the state data store may,in some examples, by a key value data store in which device staterepresentations are organized according to key values, such as deviceidentifiers or other key value information. In one particular example,when the first state is the Color=Yellow state, the actual staterepresentation for the device may be updated at operation 1512 toindicate a state of Color=Yellow, such as depicted in device staterepresentations 1205A of FIG. 12 .

At operation 1514, it is determined whether the device's assumption ofthe first state, at least in part, triggers additional processing, suchas additional processing by the device state management service. Someexample processes for determination of processing triggered based oninformation provided by a device are described in detail above withrespect to FIG. 8 and are not repeated here. In some examples, thedetermination of operation 1514 may be performed, in whole or in part,by a gateway component based on information stored in or otherwiseprovided by a device registry, such as device registry 142 as describedabove. For example, the device registry may store rules or otherinformation associated with processing operations for devices and/orclasses of devices that may be triggered based, at least in part, onvarious states, events, and/or other conditions. The device registry mayalso store information identifying various functions, such asevent-triggered processing functions, to invoke based, at least in part,on various states, events, and/or other conditions. As set forth above,the use of event-triggered computing functions may, in some examples,reduce costs and improve efficiency by consuming computing resourcesonly at times when they are invoked. In some examples, additionalprocessing may be triggered to identify errors, calculate mean, maximum,and minimum values, and other operations. As another example, additionalprocessing may be triggered to identify safety concerns and otherproblematic conditions, such as multiple readings that fall outside of adesired range.

If, at operation 1514, it is determined that additional processing hasbeen triggered, then, at operation 1516, one or more additionalprocessing operations may be performed. Some example processes forperformance of processing operations triggered based on informationprovided by a device are described in detail above with respect to FIG.8 and are not repeated here. In some examples, performance of theadditional processing may include invoking one or more functions forprocessing of the request, such as one or more event-triggeredprocessing functions. As set forth above, such functions may beidentified, at operation 1512, based on information stored in a deviceregistry. In some examples, upon completion of the additionalprocessing, a device gateway may receive an acknowledgment of processingand transmit the acknowledgement back to the device.

Upon performance of the additional processing (or if no additionalprocessing was required), the process may then proceed to operation1518, at which a request may be received for the device to assume asecond sate that is different from the first state. The request may, forexample, be received from an application, such as application 1210 ofFIGS. 12 and 13 , and received by a state management API, such as statemanagement API 1215 of FIGS. 12 and 13 . The request may, for example,identify the device using its respective device identifier and alsoprovide an indication of the desired state. Next, at operation 1520, adesired device state representation for the device may be updated toidentify the second state. In some examples, the state management APImay update a state data store that includes the desired device staterepresentations. In one particular example, when the second state is theColor=Red state, the desired state representation for the device may beupdated at operation 1520 to indicate a state of Color=Red, such asdepicted in device state representations 1205B of FIG. 12 .

At operation 1522, an event may be generated indicating that the desireddevice state representation has been updated. In some examples, theevent may be generated by a state data store in which the device staterepresentations are stored. At operation 1523, an event-triggeredprocessing function invoked to process the event by providing updateparameters for issuing of a state update command to the device. Forexample, an event triggered processing function, such as function 1225of FIG. 12 , may compare the actual and desired state representationvalues for the device and may determine that the desired state (i.e.,the second state) differs from the actual state (i.e., the first state).The event-triggered processing function may then, for example, provide,to a gateway, state change parameters indicating the device identifierfor device, the desired state (i.e., the second state), and the actualstate (i.e., the first state).

The process may then proceed to FIG. 15B, and, more specifically, tooperation 1524, at which a command may be issued to the device to assumethe second state. As set forth above, in some examples, a gateway maygenerate the state change command by converting information within thestate change parameters into a different format. For example, thegateway may convert the state change parameters 1254 from a JSONformatted message into a binary format. The gateway may send the satechange command to the device, which may, in turn, change its state tothe second state.

At operation 1526, an indication may be received associated with thedevice assuming the second state. The indication may include informationreported by the device to a device gateway using a binary format, whichmay be converted by the device format to a JSON or other format. Atoperation 1528, the actual device state representation for the devicemay be updated to indicate the second state. Operation 1528 may include,for example, any, or all, of the details described above with respect tooperation 1512, which are not repeated here. In one particular example,when the second state is the Color=Red state, the actual staterepresentation for the device may be updated at operation 1528 toindicate a state of Color=Red, such as depicted in device staterepresentations 1205C of FIG. 13 . It is further noted that, althoughnot explicitly depicted in FIG. 15B, it may be determined whether thedevice's assumption of the second state triggers additional processing,such as described above with respect to operations 1514 and 1516relative to the assumption of the first state.

As set forth above, in addition to requesting device state changes suchas described above with respect to FIGS. 15A-B, applications may alsorequest to read or otherwise obtain an indication of a state of adevice. Referring now to FIG. 16 , an example process for responding toa request for device state information will now be described. Atoperation 1610, a request for an indication of an actual state of adevice, such as a read request, may be received, for example, from anapplication. At operation 1612, a time period between the receiving ofthe request (i.e., at operation 1610) and a most recent update of theactual device state representation for the device is determined. In someexamples, when a device state representation is updated, a timestamp orother indication of the time of the update may be stored in therepresentation or at an accessible location. This timestamp or timeindication may allow the period determined at operation 1612 to beefficiently ascertained.

At operation 1614, it is determined whether the time period determinedat operation 1614 exceeds a threshold time period. In some examples, oneor more devices and/or classes of devices may have an associatedthreshold time period associated with engagement of the devices torequest updated state information. For example, this threshold timeperiod may sometimes be determined based on factors such as a frequencywith which a device changes its state and/or a time period after whichpreviously reported device states may be considered unreliable and/orunsafe. In some examples, the threshold time period for the device maybe obtained from a device registry 142 using procedures such as thosedescribed above. If the time period determined at operation 1612 doesnot exceed the threshold time period, then, at operation 1616, therequest received at operation 1610 may be processed without directlyengaging the device. For example, the request may be processed byobtaining and returning an indication of the device state that is storedin the actual device state representation without requesting updateddevice state information from the device.

If, on the other hand, the time period determined at operation 1612exceeds the threshold time period, then, at operation 1618, updatedstate information may be requested for the device. For example, thedevice may be contacted via a device gateway using procedures such asthose described above. At operation 1620, updated device information maybe received from the device. The updated device information may includean indication of a recently assumed different state or confirmation thatthe device has maintained its prior state. At operation 1622, the actualdevice state representation is updated, if necessary, based on theupdated state information. For example, if the device has recentlyassumed a different state, then the actual device state representationmay be updated to reflect the different state and to reflect an updatedtimestamp. By contrast, if the device has maintained its prior statethen the actual device state representation may continue to reflect theprior state but may update the timestamp to reflect receipt of theupdated device information. At operation 1624, the request may beresponded to by returning an indication of the device state based on theupdated state information. For example, the state reported by the devicein the updated state information (e.g., either a recently assumeddifferent state or a maintaining of a prior state) may be returned tothe requesting application.

As set forth above, the techniques described herein may allowapplications to be controlled using, for example, state change requestsand other commands. In some examples, the techniques described hereinmay also allow one or more virtual commands to be defined, exposed, andrequested for control of devices. Virtual commands may be formulatedbased on one or more existing device commands that are executable by adevice. As set forth above, in one example, a virtual “blink” commandfor a light could include combinations of existing “on” and “off”commands called in succession to generate a blinking effect. In someexamples, a virtual command may be a command associated with an updatedfirmware version that may be formulated based on one or more existingcommands associated with a prior firmware version, thereby effectivelyallowing an updated firmware version command to be issued for devicesthat are still executing the prior firmware version. Referring now toFIG. 17 , an example process for implementing of a virtual command willnow be described in detail. In particular, at operation 1710, anindication may be received of a relationship between a virtual commandand one or more existing commands, which are commands that areexecutable by a device. For example, at operation 1710, an indicationmay be received indicating that a “blink” command for a light couldinclude an alternation between ten existing “on” commands and tenexisting “off” commands in succession. In some examples, the receivedindication may specify or indicate one or more particular devices and/orclasses of devices for which the virtual command is to be exposed.

At operation 1712, one or more event-triggered processing functions maybe defined for executing the one or more existing commands, for example,based, at least in part, on their relationship with the virtual command.For example, an event-triggered processing function may be defined forthe “blink” command that alternates between execution of existing “on”commands and ten existing “off” commands in succession. At operation1714, instructions may be stored, in a device registry, such as deviceregistry 142, to implement the virtual command by invoking the one ormore event-triggered processing functions. These instructions may beassociated, in the device registry, with each device and/or class ofdevices for which the virtual command is to be exposed.

At operation 1716, the virtual command may be exposed. For example, anAPI call for requesting issuance of the virtual command may be exposedto various users of a device state management service. In some examples,the virtual command may be exposed using a directory or otherinformational organization structure that may allow users to look upvirtual commands and other device management features that are exposedfor various devices and/or classes of devices. At operation 1718, arequest to issue the virtual command is received. For example, anapplication may issue an API call that may be received by a statemanagement API. The request may, for example, provide one or more deviceidentifiers for one or more devices to which the virtual is to beissued.

At operation 1720, it may be determined, based, at least in part, on theinstructions stored in the device registry, that the requested virtualcommand will be implemented by invoking the one or more event-triggeredprocessing functions. In some examples, the state management API and/orother components may access the device registry and may provideinformation such as a device identifier, a device class identifierand/or an identifier of the virtual command in order to access thestored instructions within the device registry that identify the one ormore event-triggered processing functions. At operation 1722, the one ormore event-triggered processing functions may be invoked, for example,by the state management API and/or other components. At operation 1724,the one or more event-triggered processing functions may implement thevirtual command by issuing the related existing commands. The relatedexisting commands may be issued to one or devices identified by therequest received at operation 1718. The related existing commands may beissued based, at least in part, on the relationship between the virtualcommand and the existing commands. For example, if the requested virtualcommand is the “blink” command described above, then the one or moreevent-triggered processing functions may alternate between issuing tenexisting “on” commands and ten existing “off” commands in succession forthe identified device. The commands may be issued by, for example,providing instructions to a device gateway that may be used tocommunicate with the identified device. It is once again noted that theuse of event-triggered processing functions may, in some examples,reduce costs and improve efficiency by consuming computing resourcesonly at times when they are invoked. However, as should be appreciated,the described virtual command implementation techniques are not limitedto the use of event-triggered processing functions and one or more otherprocessing functions may be employed as an alternative, or in additionto, event-triggered processing functions.

Example Processing Architecture

Devices 130 access computing service 110 via network 120. In someembodiments, network 120 may comprise the Internet or World Wide Web andservice 110 may be considered a cloud or web service. FIGS. 18 and 19depict example operating environments that might be used to implementservice 110 and its various components. Generally, FIG. 18 depicts a webservices platform that comprises a plurality of datacenters. FIG. 19depicts a datacenter that comprises a plurality of computers.

Turning to the details of FIG. 18 , this figure depicts an example of asuitable computing environment in which embodiments described herein maybe implemented. A cloud service provider (such as web services platform1808) may configure the illustrated computing environment to hostvirtual clouds of entities and to enable communication paths betweenthese virtual clouds that may otherwise be isolated. In particular, FIG.18 is a system and network diagram that shows an illustrative operatingenvironment 1800 that includes a web services platform 1808 forimplementing virtual clouds and for providing on-demand access tocomputer resources, such as virtual machine instances. Web servicesplatform 1808 can provide computer resources for executing applicationson a permanent or an as-needed basis and may be configured as a privatenetwork. These computer resources may include various types ofresources, such as data processing resources, data storage resources,data communication resources, and the like. Each type of computerresource may be general-purpose or may be available in a number ofspecific configurations. For example, data processing resources may beavailable as virtual machine instances. The instances may be configuredto execute applications, including web servers, application servers,media servers, database servers, and the like. Data storage resourcesmay include file storage devices, block storage devices, and the like.

Each type or configuration of computer resource may be available indifferent sizes, such as large resources consisting of many processors,large amounts of memory, and/or large storage capacity, and smallresources consisting of fewer processors, smaller amounts of memory,and/or smaller storage capacity. Entities may choose to allocate anumber of small processing resources as web servers and/or one largeprocessing resource as a database server, for example.

The computer resources provided by web services platform 1808 may beenabled by one or more datacenters 1802A-1802N, which may be referredherein singularly as “datacenter 1802” or in the plural as “datacenters1802.” Datacenters 1802 may be facilities that house and operatecomputer systems and associated components and may include redundant andbackup power, communications, cooling, and security systems. Datacenters1802 may be located in a same geographical area, such as in a samefacility, and may be interconnected using private networks, such ashigh-speed fiber optic networks, controlled and managed by a serviceprovider of web services platform 1808. Datacenters 1802 may also bedistributed across geographically disparate locations and may beinterconnected in part using public networks, such as the Internet. Oneillustrative configuration for datacenter 1802 that implements theconcepts and technologies disclosed herein is described below withregard to FIG. 19 .

Entities of web services platform 1808 may access the computer resourcesprovided by datacenters 1802 over a Wide Area Network (WAN) 1806.Although a WAN is illustrated in FIG. 18 , it should be appreciated thata Local Area Network (LAN), the Internet, or any other networkingtopology known in the art that connects datacenters 1802 to remoteentities and other users may be utilized. It should also be appreciatedthat combinations of such networks may also be utilized.

An entity or other entities that are users of web services platform 1808may utilize a computing system 1804 to access the computer resourcesprovided by datacenters 1802. User computing system 1804 comprises acomputer capable of accessing web services platform 1808, such as aserver computer, a desktop or laptop personal computer, a tabletcomputer, a wireless telephone, a PDA, an e-reader, a game console, aset-top box, an automobile, an automobile system, a home appliance, amanufacturing device or sensor, a building control system, a farminginstrument, or any other computing node or thing that is able tocommunicate with data center 1802. In an example scenario, computingsystem 1804 may correspond, for example, devices 130 described above.

In an example scenario, a particular user computing system 1804 may beutilized to configure aspects of the computer resources provided by webservices platform 1808. In this regard, web services platform 1808 mayprovide a web interface through which aspects of its operation may beconfigured through the use of a web browser application programexecuting on user computing system 1804. Alternatively, a stand-aloneapplication program executing on user computing system 1804 may accessan application programming interface (API) exposed by web servicesplatform 1808 for performing the configuration operations. Othermechanisms for configuring the operation of web services platform 1808,including launching new virtual machine instances on web servicesplatform 1808, may also be utilized.

According to embodiments disclosed herein, capacities of purchasedcomputer resources provided by web services platform 1808 can be scaledin response to demand. In this regard, scaling refers to the process ofinstantiating, which may also be referred to herein as “launching” or“creating,” or terminating, which may also be referred to herein as“de-scaling,” instances of computer resources in response to demand.

Web services platform 1808 may also be configured with a deploymentcomponent to assist entities in the deployment of new instances ofcomputer resources. The deployment component may receive a configurationfrom an entity that may include data describing how new instances shouldbe configured. For example, the configuration may specify one or moreapplications or software components that should be installed in newinstances, provide scripts and/or other types of code to be executed innew instances, provide cache warming logic specifying how an applicationcache should be prepared, and other types of information. The deploymentcomponent utilizes the entity-provided configuration and cache warminglogic to launch, configure, and prime new instances of computerresources.

FIG. 19 depicts a computing system diagram that illustrates oneconfiguration for datacenter 1802 that implements web services platform1808. The example datacenter 1802 shown in FIG. 19 may include severalserver computers 1902A-1902N, which may be referred herein singularly as“server computer 1902” or in the plural as “server computers 1902,” forproviding computer resources for hosting virtual clouds and forexecuting applications. Server computers 1902 may be standard tower orrack-mount server computers configured appropriately for providing thecomputer resources described above. For instance, in one implementationserver computers 1902 may be configured to provide instances 1906A-1906Nof computer resources.

Instances 1906A-1906N, which may be referred herein singularly as“instance 1906” or in the plural as “instances 1906,” may be virtualmachine instances. As known in the art, a virtual machine instance is aninstance of a software implementation of a machine (i.e., a computer)that executes programs like a physical machine. In the example ofvirtual machine instances, each server 1902 may be configured to executean instance manager 1908 capable of executing the instances. Instancemanager 1908 may be a hypervisor or another type of program configuredto enable the execution of multiple instances 1906 on a single server1902, for example. As discussed above, each of instances 1906 may beconfigured to execute all or a portion of an application.

It should be appreciated that although the embodiments disclosed hereinare described primarily in the context of virtual machine instances,other types of instances can be utilized with the concepts andtechnologies disclosed herein. For instance, the technologies disclosedherein may be utilized with instances of storage resources, instances ofdata communications resources, and with other types of resources. Theembodiments disclosed herein may also execute all or a portion of anapplication directly on a computer system without utilizing virtualmachine instances.

Datacenter 1802 shown in FIG. 19 may also include a server computer 1904reserved for executing software components for managing the operation ofdatacenter 1802, server computers 1902, and instances 1906. Inparticular, server computer 1904 may execute a management component1910. As discussed above, working between FIGS. 18 and 19 , an entity ofweb services platform 1808 may utilize user computing system 1804 toaccess management component 1910 to configure various aspects of theoperation of web services platform 1808 and instances 1906 purchased bythe entity. For example, the entity may purchase instances and makechanges to the configuration of the instances. The entity may alsospecify settings regarding how the purchased instances are to be scaledin response to demand. The entity may also provide requests to launchinstances to management component 1610.

As also described briefly above, an auto scaling component 1912 mayscale instances 1906 based upon rules defined by an entity of webservices platform 1908. For example, auto scaling component 1912 mayallow an entity to specify scale up rules for use in determining whennew instances should be instantiated and scale down rules for use indetermining when existing instances should be terminated.

As discussed briefly above, datacenter 1802 may also be configured witha deployment component 1914 to assist entities in the deployment of newinstances 1906 of computer resources. Deployment component 1914 mayreceive a configuration from an entity that includes data describing hownew instances 1906 should be configured. For example, the configurationmay specify one or more applications that should be installed in newinstances 1906, provide scripts and/or other types of code to beexecuted for configuring new instances 1906, provide cache warming logicspecifying how an application cache should be prepared, and other typesof information.

Deployment component 1914 may utilize the entity-provided configurationand cache warming logic to configure, prime, and launch new instances1906. The configuration, cache warming logic, and other information maybe specified by an entity using management component 1910 or byproviding this information directly to deployment component 1914. Othermechanisms may also be utilized to configure the operation of deploymentcomponent 1914.

In the example datacenter 1802 shown in FIG. 19 , an appropriate LAN1916 may be utilized to interconnect server computers 1902A-1902N andserver computer 1904. LAN 1916 may also be connected to WAN 1806illustrated in FIG. 18 . It should be appreciated that the networktopology illustrated in FIGS. 18 and 19 has been greatly simplified andthat many more networks and networking devices may be utilized tointerconnect the various computing systems disclosed herein. Appropriateload balancing devices or software modules may also be utilized forbalancing a load between each of datacenters 1802A-1802N, between eachof server computers 1902A-1902N in each datacenter 1802 and betweeninstances 1906 purchased by each entity of web services platform 1808.These network topologies and devices should be apparent to those skilledin the art.

It should be appreciated that datacenter 1802 described in FIG. 19 ismerely illustrative and that other implementations may be utilized. Inparticular, functionality described herein as being performed bymanagement component 1910, auto scaling component 1912, and deploymentcomponent 1914 may be performed by one another, may be performed byother components, or may be performed by a combination of these or othercomponents. Additionally, it should be appreciated that thisfunctionality may be implemented in software, hardware, or a combinationof software and hardware. Other implementations should be apparent tothose skilled in the art.

FIG. 20 depicts an example computer architecture for a computing system2000 capable of executing software for performing operations asdescribed above in connection with FIGS. 1-19 . The computerarchitecture shown in FIG. 20 illustrates a conventional servercomputer, workstation, desktop computer, laptop, tablet, networkappliance, PDA, e-reader, digital cellular phone, or other computingnode, and may be utilized to execute any aspects of the softwarecomponents presented herein described as executing on servers 140, 142,144, 146, 150, 220, 230, and 240, on devices 130, within datacenters1802A-1802N, on server computers 1902A-1902N, or on any other computingsystem mentioned herein.

Computer 2000 may include a baseboard, or “motherboard,” which is aprinted circuit board to which a multitude of components or devices maybe connected by way of a system bus or other electrical communicationpaths. One or more central processing units (CPUs) 2004 may operate inconjunction with a chipset 2006. CPUs 2004 may be standard programmableprocessors that perform arithmetic and logical operations necessary forthe operation of computer 2000.

CPUs 2004 may perform the necessary operations by transitioning from onediscrete physical state to the next through the manipulation ofswitching elements that differentiate between and change these states.Switching elements may generally include electronic circuits thatmaintain one of two binary states, such as flip-flops, and electroniccircuits that provide an output state based on the logical combinationof the states of one or more other switching elements, such as logicgates. These basic switching elements may be combined to create morecomplex logic circuits including registers, adders-subtractors,arithmetic logic units, floating-point units, and the like.

Chipset 2006 may provide an interface between CPUs 2004 and theremainder of the components and devices on the baseboard. Chipset 2006may provide an interface to a random access memory (RAM) 2008 used asthe main memory in computer 2000. Chipset 2006 may further provide aninterface to a computer-readable storage medium, such as a read-onlymemory (ROM) 2020 or non-volatile RAM (NVRAM) (not shown), for storingbasic routines that may help to start up computer 2000 and to transferinformation between the various components and devices. ROM 2020 orNVRAM may also store other software components necessary for theoperation of computer 2000 in accordance with the embodiments describedherein.

Computer 2000 may operate in a networked environment using logicalconnections to remote computing nodes and computer systems through LAN2016. Chipset 2006 may include functionality for providing networkconnectivity through a network interface controller (NIC) 2022, such asa gigabit Ethernet adapter. NIC 2022 may be capable of connecting thecomputer 2000 to other computing nodes over LAN 2016. It should beappreciated that multiple NICs 2022 may be present in computer 2000,connecting the computer to other types of networks and remote computersystems.

Computer 2000 may be connected to a mass storage device 2028 thatprovides non-volatile storage for the computer. Mass storage device 2028may store system programs, application programs, other program modules,and data, which have been described in greater detail herein. Massstorage device 2028 may be connected to computer 2000 through a storagecontroller 2024 connected to chipset 2006. Mass storage device 2028 mayconsist of one or more physical storage units. Storage controller 2024may interface with the physical storage units through a serial attachedSCSI (SAS) interface, a serial advanced technology attachment (SATA)interface, a fiber channel (FC) interface, or other type of interfacefor physically connecting and transferring data between computers andphysical storage units.

Computer 2000 may store data on mass storage device 2028 by transformingthe physical state of the physical storage units to reflect theinformation being stored. The specific transformation of a physicalstate may depend on various factors and on different implementations ofthis description. Examples of such factors may include, but are notlimited to, the technology used to implement the physical storage unitsand whether mass storage device 2028 is characterized as primary orsecondary storage and the like.

For example, computer 2000 may store information to mass storage device2028 by issuing instructions through storage controller 2024 to alterthe magnetic characteristics of a particular location within a magneticdisk drive unit, the reflective or refractive characteristics of aparticular location in an optical storage unit, or the electricalcharacteristics of a particular capacitor, transistor, or other discretecomponent in a solid-state storage unit. Other transformations ofphysical media are possible without departing from the scope and spiritof the present description, with the foregoing examples provided only tofacilitate this description. Computer 2000 may further read informationfrom mass storage device 2028 by detecting the physical states orcharacteristics of one or more particular locations within the physicalstorage units.

In addition to mass storage device 2028 described above, computer 2000may have access to other computer-readable storage media to store andretrieve information, such as program modules, data structures, or otherdata. It should be appreciated by those skilled in the art thatcomputer-readable storage media can be any available media that providesfor the storage of non-transitory data and that may be accessed bycomputer 2000.

By way of example and not limitation, computer-readable storage mediamay include volatile and non-volatile, transitory computer-readablestorage media and non-transitory computer-readable storage media,removable and non-removable media implemented in any method ortechnology. Computer-readable storage media includes, but is not limitedto, RAM, ROM, erasable programmable ROM (EPROM), electrically erasableprogrammable ROM (EEPROM), flash memory or other solid-state memorytechnology, compact disc ROM (CD-ROM), digital versatile disk (DVD),high definition DVD (HD-DVD), BLU-RAY, or other optical storage,magnetic cassettes, magnetic tape, magnetic disk storage, other magneticstorage devices, or any other medium that can be used to store thedesired information in a non-transitory fashion.

Mass storage device 2028 may store an operating system utilized tocontrol the operation of the computer 2000. According to one embodiment,the operating system comprises a version of the LINUX operating system.According to another embodiment, the operating system comprises aversion of the WINDOWS SERVER operating system from the MICROSOFTCorporation. According to further embodiments, the operating system maycomprise a version of the UNIX operating system. It should beappreciated that other operating systems may also be utilized. Massstorage device 2028 may store other system or application programs anddata utilized by computer 2000, such as management component 2010 and/orthe other software components described above.

Mass storage device 2028 or other computer-readable storage media mayalso be encoded with computer-executable instructions, which, whenloaded into computer 2000, transforms the computer from ageneral-purpose computing system into a special-purpose computer capableof implementing the embodiments described herein. Thesecomputer-executable instructions transform computer 2000 by specifyinghow CPUs 2004 transition between states, as described above. Computer2000 may have access to computer-readable storage media storingcomputer-executable instructions, which, when executed by computer 2000,may perform operating procedures described above in connection withFIGS. 1-10 .

Computer 2000 may also include an input/output controller 2032 forreceiving and processing input from a number of input devices, such as akeyboard, a mouse, a touchpad, a touch screen, an electronic stylus, orother type of input device. Similarly, input/output controller 2032 mayprovide output to a display, such as a computer monitor, a flat-paneldisplay, a digital projector, a printer, a plotter, or other type ofoutput device. It will be appreciated that computer 2000 may not includeall of the components shown in FIG. 20 , may include other componentsthat are not explicitly shown in FIG. 20 , or may utilize anarchitecture completely different than that shown in FIG. 20 .

As described herein, a computing node may be a physical computing node,such as computer 2000 of FIG. 20 . A computing node may also be avirtual computing node, such as a virtual machine instance, or a sessionhosted by a physical computing node, where the computing node isconfigured to host one or more sessions concurrently.

It should be appreciated that the subject matter presented herein may beimplemented as a computer process, a computer-controlled apparatus, or acomputing system or an article of manufacture, such as acomputer-readable storage medium. While the subject matter describedherein is presented in the general context of program modules thatexecute on one or more computing devices, those skilled in the art willrecognize that other implementations may be performed in combinationwith other types of program modules. Generally, program modules includeroutines, programs, components, data structures, and other types ofstructures that perform particular tasks or implement particularabstract data types.

Those skilled in the art will also appreciate that the subject matterdescribed herein may be practiced on or in conjunction with othercomputer system configurations beyond those described herein, includingmultiprocessor systems, microprocessor-based or programmable consumerelectronics, minicomputers, mainframe computers, handheld computers,personal digital assistants, e-readers, cellular telephone devices,special purposed hardware devices, network appliances, and the like. Theembodiments described herein may also be practiced in distributedcomputing environments, where tasks are performed by remote processingdevices that are linked through a communications network. In adistributed computing environment, program modules may be located inboth local and remote memory storage devices.

Networks established by or on behalf of an entity, such as a company ororganization, to provide one or more services (such as various types ofcloud-based computing or storage) accessible via the Internet and/orother networks to a distributed set of clients may be termed providernetworks. Such a provider network may include one or more data centershosting various resource pools, such as collections of physical and/orvirtualized computer servers, storage devices, networking equipment, andthe like, that may be used to implement and distribute theinfrastructure and services offered by the provider network. Theresources may in some embodiments be offered to clients in units called“instances,” such as virtual or physical compute instances or storageinstances. A virtual computing instance may, for example, comprise oneor more servers with a specified computational capacity (which may bespecified by indicating the type and number of CPUs, the main memorysize, and so on) and a specified software stack (e.g., a particularversion of an operating system, which may in turn run on top of ahypervisor).

A number of different types of computing devices may be used singly orin combination to implement the resources of the provider network indifferent embodiments, including general purpose or special purposecomputer servers, storage devices, network devices, and the like. Insome embodiments a client or user may be provided direct access to aresource instance, e.g., by giving a user an administrator login andpassword. In other embodiments, the provider network operator may allowclients to specify execution requirements for specified clientapplications and schedule execution of the applications on behalf of theclient on execution platforms (such as application server instances,Java™ virtual machines (JVMs), general purpose or special purposeoperating systems, platforms that support various interpreted orcompiled programming languages, such as Ruby, Perl, Python, C, C++, andthe like or high-performance computing platforms) suitable for theapplications. This may be done without, for example, requiring theclient to access an instance or an execution platform directly. A givenexecution platform may utilize one or more resource instances in someimplementations; in other implementations, multiple execution platformsmay be mapped to a single resource instance.

In many environments, operators of provider networks that implementdifferent types of virtualized computing, storage, and/or othernetwork-accessible functionality may allow customers to reserve orpurchase access to resources in various resource acquisition modes. Thecomputing resource provider may provide facilities for customers toselect and launch the desired computing resources, deploy applicationcomponents to the computing resources, and maintain an applicationexecuting in the environment. In addition, the computing resourceprovider may provide further facilities for the customer to quickly andeasily scale up or scale down the numbers and types of resourcesallocated to the application, either manually or through automaticscaling, as demand for or capacity requirements of the applicationchange. The computing resources provided by the computing resourceprovider may be made available in discrete units, which may be referredto as instances. An instance may represent a physical server hardwareplatform, a virtual machine instance executing on a server, or somecombination of the two. Various types and configurations of instancesmay be made available, including different sizes of resources executingdifferent operating systems (OS) and/or hypervisors and with variousinstalled software applications, runtimes, and the like. Instances mayfurther be available in specific availability zones, representing a datacenter or other geographic location of the underlying computinghardware, for example.

In some embodiments the provider network may be organized into aplurality of geographical regions, and each region may include one ormore availability zones. An availability zone (which may also bereferred to as an availability container) in turn may comprise one ormore distinct locations or data centers, configured in such a way thatthe resources in a given availability zone may be isolated or insulatedfrom failures in other availability zones. That is, a failure in oneavailability zone may not be expected to result in a failure in anyother availability zone. Thus, the availability profile of a resourceinstance is intended to be independent of the availability profile of aresource instance in a different availability zone. Clients may be ableto protect their applications from failures at a single location bylaunching multiple application instances in respective availabilityzones. At the same time, in some implementations, inexpensive and lowlatency network connectivity may be provided between resource instancesthat reside within the same geographical region (and networktransmissions between resources of the same availability zone may beeven faster).

The provider network may make instances available “on-demand,” allowinga customer to select a number of instances of a specific type andconfiguration (e.g. size, platform, tenancy, availability zone, and thelike) and quickly launch the instances for deployment. On-demandinstances may further be added or removed as needed, either manually orautomatically through auto scaling, as demand for or capacityrequirements changes over time. The customer may incur ongoing usagecosts related to their on-demand instances, based on the number of hoursof operation and/or the actual resources utilized, for example.

The computing resource provider may also make reserved instancesavailable to the customer. Reserved instances may provide the customerwith the ability to reserve a number of a specific type andconfiguration of instances for a fixed term, such as one year or threeyears, for a low, upfront cost in exchange for reduced hourly or otherusage costs, for example, if and when the instances are launched. Thismay allow the customer to defer costs related to scaling up the deployedapplication in response to increase in demand, while ensuring that theright resources will be available when needed. While reserved instancesprovide customers with reliable, stand-by capacity for scaling of theirapplication, purchasing reserved instances may also lock the customerinto a specific number, type, and/or configuration of computing resourcein a specific availability zone for a longer period than desired. If thetechnical architecture or needs of the application change, the customermay not be able to realize a return on the customer's investment in thereserved instances.

Operators of such provider networks may in some instances implement aflexible set of resource reservation, control, and access interfaces fortheir clients. For example, a resource manager of the provider networkmay implement a programmatic resource reservation interface (e.g., via aweb site or a set of web pages) that allows clients to learn about,select, purchase access to and/or reserve resource instances. In someembodiments discussed below where an entity, such as a resource manageror a pricing optimizer, is described as implementing one or moreprogrammatic interfaces, such as a web page or an API, an interfacemanager subcomponent of that entity may be responsible for theinterface-related functionality. In many embodiments equivalentinterface-related functionality may be implemented by a separate orstandalone interface manager, external to the resource manager and thepricing optimizer. Such an interface may include capabilities to allowbrowsing of a resource catalog and details and specifications of thedifferent types or sizes of resources supported and the differentreservation types or modes supported, pricing models, and so on.

The provider network may support several different purchasing modes(which may also be referred to herein as reservation modes)corresponding to some of the instance types described above. Forexample, the provider network may support long-term reservations,on-demand resource allocations, or spot-price-based resourceallocations. Using the long-term reservation mode, a client may make alow, one-time, upfront payment for a resource instance, reserve it for aspecified duration, such as a one- or three-year term, and pay a lowhourly rate for the instance; the client would be assured of having thereserved instance available for the term of the reservation. Using theon-demand mode, a client could pay for capacity by the hour (or someappropriate time unit), without any long-term commitments or upfrontpayments. In the spot-price mode, a client could specify the maximumprice per unit of time that it is willing to pay for a particular typeof resource, and if the client's maximum price exceeded a dynamic spotprice determined at least in part by supply and demand, then that typeof resource would be provided to the client. In some embodiments,dynamically resizable pools of resource instances may be set aside forthe different reservation types or modes—e.g., long-term reservedinstances may be allocated from one pool, on-demand instances fromanother, and so on. During periods when the supply of the requestedresource type exceeded the demand, the spot price may becomesignificantly lower than the price for the on-demand mode. In someimplementations, if the spot price increases beyond the maximum bidspecified by a client, a resource allocation may be interrupted—i.e., aresource instance that was previously allocated to the client may bereclaimed by the resource manager and may be allocated to some otherclient that is willing to pay a higher price. Other purchasing modes orcombinations of modes may be implemented by the resource manager in someembodiments.

It will be appreciated that while illustrative embodiments have beendisclosed, the scope of potential embodiments is not limited to thoseexplicitly described. For example, while the concepts are described withreference to requests received to perform particular types of functionsor commands, the envisioned embodiments extend to processing involvingany and all types of functions and commands. Similarly, while theconcepts are described with reference to particular protocols andformats, the envisioned embodiments extend to processing involving anyand all types of protocols and formats.

Each of the processes, methods, and algorithms described in thepreceding sections may be embodied in, and fully or partially automatedby, code modules executed by one or more computers or computerprocessors. The code modules may be stored on any type of non-transitorycomputer-readable medium or computer storage device, such as harddrives, solid state memory, optical disc, and/or the like. The processesand algorithms may be implemented partially or wholly inapplication-specific circuitry. The results of the disclosed processesand process steps may be stored, persistently or otherwise, in any typeof non-transitory computer storage such as, e.g., volatile ornon-volatile storage.

The various features and processes described above may be usedindependently of one another, or may be combined in various ways. Allpossible combinations and subcombinations are intended to fall withinthe scope of this disclosure. In addition, certain method or processblocks may be omitted in some implementations. The methods and processesdescribed herein are also not limited to any particular sequence, andthe blocks or states relating thereto can be performed in othersequences that are appropriate. For example, described blocks or statesmay be performed in an order other than that specifically disclosed, ormultiple blocks or states may be combined in a single block or state.The example blocks or states may be performed in serial, in parallel, orin some other manner. Blocks or states may be added to or removed fromthe disclosed example embodiments. The example systems and componentsdescribed herein may be configured differently than described. Forexample, elements may be added to, removed from or rearranged comparedto the disclosed example embodiments.

It will also be appreciated that various items are illustrated as beingstored in memory or on storage while being used, and that these items orportions of thereof may be transferred between memory and other storagedevices for purposes of memory management and data integrity.Alternatively, in other embodiments some or all of the software modulesand/or systems may execute in memory on another device and communicatewith the illustrated computing systems via inter-computer communication.Furthermore, in some embodiments, some or all of the systems and/ormodules may be implemented or provided in other ways, such as at leastpartially in firmware and/or hardware, including, but not limited to,one or more application-specific integrated circuits (ASICs), standardintegrated circuits, controllers (e.g., by executing appropriateinstructions, and including microcontrollers and/or embeddedcontrollers), field-programmable gate arrays (FPGAs), complexprogrammable logic devices (CPLDs), etc. Some or all of the modules,systems and data structures may also be stored (e.g., as softwareinstructions or structured data) on a computer-readable medium, such asa hard disk, a memory, a network, or a portable media article to be readby an appropriate drive or via an appropriate connection. The systems,modules, and data structures may also be transmitted as generated datasignals (e.g., as part of a carrier wave or other analog or digitalpropagated signal) on a variety of computer-readable transmission media,including wireless-based and wired/cable-based media, and may take avariety of forms (e.g., as part of a single or multiplexed analog signalor as multiple discrete digital packets or frames). Such computerprogram products may also take other forms in other embodiments.Accordingly, the present invention may be practiced with other computersystem configurations.

Conditional language used herein, such as, among others, “can,” “could,”“might,” “may,” “e.g.,” and the like, unless specifically statedotherwise, or otherwise understood within the context as used, isgenerally intended to convey that certain embodiments include, whileother embodiments do not include, certain features, elements, and/orsteps. Thus, such conditional language is not generally intended toimply that features, elements, and/or steps are in any way required forone or more embodiments or that one or more embodiments necessarilyinclude logic for deciding, with or without author input or prompting,whether these features, elements and/or steps are included or are to beperformed in any particular embodiment. The terms “comprising,”“including,” “having,” and the like are synonymous and are usedinclusively, in an open-ended fashion, and do not exclude additionalelements, features, acts, operations, and so forth. Also, the term “or”is used in its inclusive sense (and not in its exclusive sense) so thatwhen used, for example, to connect a list of elements, the term “or”means one, some, or all of the elements in the list.

While certain example embodiments have been described, these embodimentshave been presented by way of example only and are not intended to limitthe scope of the inventions disclosed herein. Thus, nothing in theforegoing description is intended to imply that any particular feature,characteristic, step, module, or block is necessary or indispensable.Indeed, the novel methods and systems described herein may be embodiedin a variety of other forms; furthermore, various omissions,substitutions, and changes in the form of the methods and systemsdescribed herein may be made without departing from the spirit of theinventions disclosed herein. The accompanying claims and theirequivalents are intended to cover such forms or modifications as wouldfall within the scope and spirit of certain of the inventions disclosedherein.

What is claimed is:
 1. A system for operating a plurality of devices ina network, comprising: at least one processor; a data store maintainingdata representing the plurality of devices; a memory in communicationwith the at least one processor, the memory storing instructions thatupon execution by the at least one processor carry out operationscomprising: receiving data indicative of a change to a first deviceamong the plurality of devices; updating the data representing a firstdevice stored in the data store to reflect the data indicative of thechange to the first device; and communicating to the first device sothat the data representing the first device stored in the data storereflects an operation of the first device, wherein the first deviceupdates its state to reflect the operation stored in the data store. 2.The system as recited in claim 1 wherein the data representing the firstdevice comprises at least one state of the first device.
 3. The systemas recited in claim 1 wherein the data indicative of a change representsa change to one of an actual state or desired state.
 4. The system asrecited in claim 1 wherein the memory storing instructions that uponexecution on the at least one processor carry out the operationscomprising communicating with the first device comprises exchanging aJavaScript Object Notation (JSON) document.
 5. The system as recited inclaim 1 wherein the memory storing instructions that upon execution onthe at least one processor carry out the operations comprisingcommunicating with the first device comprises communicating a pluralityof instructions to dynamically change the state of the device.
 6. Thesystem as recited in claim 1 wherein the memory storing instructionsthat upon execution on the at least one processor node carry out theoperations comprising receiving data indicative of a change to a firstdevice among the plurality of devices comprises receiving the dataindicative of a change from the first device.
 7. The system as recitedin claim 3 wherein the memory storing instructions that upon executionon the at least one processor carry out the operations comprisingcommunicating with the first device comprises issuing one or moreinstructions to the first device based on the desired state.
 8. Anon-transitory computer-readable medium, the computer-readable mediumstoring instructions that upon execution configure a computing node atleast to: receive information indicative of a change to at least onedevice among a plurality of devices; update stored data representing theat least one device to reflect the information indicative of the changeto the at least one device; and synchronize with the at least one deviceso that the data representing the first device stored in the data storemirrors corresponding changes to the at least one device, and whereinthe first device is changed to reflect the changes stored in the datastore.
 9. The non-transitory computer-readable medium as recited inclaim 8 wherein the data representing the at least one device comprisesat least one state of the at least one device.
 10. The non-transitorycomputer-readable medium as recited in claim 8 wherein the informationindicative of a change represents a change to one of an actual state ordesired state.
 11. The non-transitory computer-readable medium asrecited in claim 8 wherein the instructions that upon executionconfigure a computing node at least to synchronize with the at least onedevice comprises transmitting a JavaScript Object Notation (JSON)document.
 12. The non-transitory computer-readable medium as recited inclaim 8 wherein the memory storing instructions that upon execution onthe at least one computing node carry out the operations comprisingsynchronizing with the at least one device comprises communicating aplurality of instructions to dynamically change the operation of thedevice.
 13. The non-transitory computer-readable medium as recited inclaim 8 wherein the instructions that upon execution configure acomputing node at least to receive information indicative of a change tothe at least one device among the plurality of devices comprisesreceiving the data indicative of a change from the at least one device.14. The non-transitory computer-readable medium as recited in claim 8wherein the memory storing instructions that upon execution on the atleast one computing node carry out the operations comprisingsynchronizing with the at least one device comprises intermittentlyconnecting with the at least one device.
 15. A method for operating aplurality of devices in a network, comprising: receiving informationindicative of a change to at least one device among a plurality ofdevices; updating stored data representing the at least one device toreflect the information indicative of the change to the at least onedevice; and synchronizing with the at least one device so that the datarepresenting the first device stored in the data store mirrorscorresponding changes to the at least one device, and wherein the firstdevice is changed to reflect the changes stored in the data store. 16.The method as recited in claim 15 wherein the data representing the atleast one device comprises at least one state of the at least onedevice.
 17. The method as recited in claim 15 wherein the informationindicative of a change represents a change to one of an actual state ordesired state.
 18. The method as recited in claim 15 wherein theinstructions that upon execution configure a computing node at least tosynchronize with the at least one device comprises transmitting aJavaScript Object Notation (JSON) document.
 19. The method as recited inclaim 15 wherein the memory storing instructions that upon execution onthe at least one computing node carry out the operations comprisingsynchronizing with the at least one device comprises communicating aplurality of instructions to dynamically change the operation of thedevice.
 20. The method as recited in claim 15 wherein the instructionsthat upon execution configure a computing node at least to receiveinformation indicative of a change to the at least one device among theplurality of devices comprises receiving the data indicative of a changefrom the at least one device.